Oct 30 22:05:11.343159 ip-10-0-131-94 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Oct 30 22:05:11.343169 ip-10-0-131-94 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Oct 30 22:05:11.343176 ip-10-0-131-94 systemd[1]: kubelet.service: Failed with result 'resources'. Oct 30 22:05:11.343409 ip-10-0-131-94 systemd[1]: Failed to start Kubernetes Kubelet. Oct 30 22:05:21.592367 ip-10-0-131-94 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Oct 30 22:05:21.592380 ip-10-0-131-94 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 458000f1c64c476bb69db4ec70cc87b2 -- Oct 30 22:07:21.852338 ip-10-0-131-94 systemd[1]: Starting Kubernetes Kubelet... Oct 30 22:07:22.233489 ip-10-0-131-94 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 22:07:22.233489 ip-10-0-131-94 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 22:07:22.233489 ip-10-0-131-94 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 22:07:22.233489 ip-10-0-131-94 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 22:07:22.237609 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237518 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:07:22.237609 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237606 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:07:22.237609 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237610 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:07:22.237609 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237613 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:07:22.237609 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237616 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:07:22.237609 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237619 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:07:22.237828 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237623 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:07:22.237828 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237626 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:07:22.237828 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237628 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:07:22.237828 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237632 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:07:22.237828 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237635 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:07:22.237828 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237637 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:07:22.237828 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237640 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:07:22.237828 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237643 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:07:22.237828 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237646 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:07:22.237828 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237649 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:07:22.237828 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237651 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:07:22.237828 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237654 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:07:22.237828 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237657 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:07:22.237828 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237659 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:07:22.237828 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237670 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:07:22.237828 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237682 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:07:22.237828 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237685 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:07:22.237828 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237688 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:07:22.237828 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237690 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:07:22.237828 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237693 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:07:22.238335 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237695 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:07:22.238335 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237698 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:07:22.238335 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237701 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:07:22.238335 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237704 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:07:22.238335 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237706 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:07:22.238335 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237710 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:07:22.238335 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237713 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:07:22.238335 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237715 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:07:22.238335 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237718 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:07:22.238335 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237721 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:07:22.238335 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237723 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:07:22.238335 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237726 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:07:22.238335 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237729 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:07:22.238335 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237732 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:07:22.238335 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237735 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:07:22.238335 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237738 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:07:22.238335 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237740 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:07:22.238335 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237743 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:07:22.238335 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237745 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:07:22.238788 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237748 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:07:22.238788 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237751 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:07:22.238788 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237753 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:07:22.238788 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237756 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:07:22.238788 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237759 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:07:22.238788 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237761 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:07:22.238788 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237764 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:07:22.238788 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237766 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:07:22.238788 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237769 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:07:22.238788 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237772 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:07:22.238788 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237774 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:07:22.238788 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237777 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:07:22.238788 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237779 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:07:22.238788 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237782 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:07:22.238788 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237784 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:07:22.238788 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237787 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:07:22.238788 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237789 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:07:22.238788 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237793 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:07:22.238788 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237797 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:07:22.239272 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237802 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:07:22.239272 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237806 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:07:22.239272 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237808 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:07:22.239272 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237812 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:07:22.239272 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237816 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:07:22.239272 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237819 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:07:22.239272 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237823 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:07:22.239272 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237826 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:07:22.239272 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237829 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:07:22.239272 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237832 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:07:22.239272 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237835 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:07:22.239272 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237838 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:07:22.239272 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237840 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:07:22.239272 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237843 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:07:22.239272 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237846 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:07:22.239272 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237848 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:07:22.239272 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237851 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:07:22.239272 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237861 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:07:22.239272 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237864 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:07:22.239767 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237868 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:07:22.239767 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237872 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:07:22.239767 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237874 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:07:22.239767 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237877 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:07:22.239767 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237880 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:07:22.239767 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237883 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:07:22.239767 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237885 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:07:22.239767 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237888 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:07:22.239767 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237890 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:07:22.239767 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237893 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:07:22.239767 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237896 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:07:22.239767 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237898 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:07:22.239767 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237901 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:07:22.239767 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237903 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:07:22.239767 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237906 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:07:22.239767 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237909 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:07:22.239767 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237911 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:07:22.239767 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237917 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:07:22.239767 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237920 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:07:22.239767 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237922 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:07:22.240258 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237926 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:07:22.240258 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237929 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:07:22.240258 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.237931 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:07:22.240258 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238797 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:07:22.240258 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238804 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:07:22.240258 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238808 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:07:22.240258 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238811 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:07:22.240258 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238814 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:07:22.240258 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238817 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:07:22.240258 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238820 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:07:22.240258 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238823 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:07:22.240258 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238825 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:07:22.240258 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238828 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:07:22.240258 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238831 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:07:22.240258 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238834 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:07:22.240258 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238837 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:07:22.240258 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238840 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:07:22.240258 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238843 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:07:22.240258 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238846 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:07:22.240711 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238849 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:07:22.240711 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238851 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:07:22.240711 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238856 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:07:22.240711 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238860 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:07:22.240711 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238863 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:07:22.240711 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238867 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:07:22.240711 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238870 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:07:22.240711 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238873 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:07:22.240711 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238876 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:07:22.240711 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238880 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:07:22.240711 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238885 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:07:22.240711 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238888 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:07:22.240711 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238891 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:07:22.240711 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238895 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:07:22.240711 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238898 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:07:22.240711 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238900 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:07:22.240711 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238903 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:07:22.240711 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238906 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:07:22.240711 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238908 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:07:22.241188 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238911 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:07:22.241188 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238914 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:07:22.241188 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238916 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:07:22.241188 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238919 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:07:22.241188 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238922 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:07:22.241188 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238924 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:07:22.241188 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238927 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:07:22.241188 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238929 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:07:22.241188 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238932 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:07:22.241188 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238935 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:07:22.241188 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238938 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:07:22.241188 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238941 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:07:22.241188 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238943 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:07:22.241188 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238946 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:07:22.241188 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238949 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:07:22.241188 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238951 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:07:22.241188 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238954 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:07:22.241188 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238957 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:07:22.241188 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238959 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:07:22.241188 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238962 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:07:22.241687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238964 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:07:22.241687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238967 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:07:22.241687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238970 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:07:22.241687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238974 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:07:22.241687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238976 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:07:22.241687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238979 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:07:22.241687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238983 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:07:22.241687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238985 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:07:22.241687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238988 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:07:22.241687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238990 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:07:22.241687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238993 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:07:22.241687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238996 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:07:22.241687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.238998 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:07:22.241687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239001 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:07:22.241687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239003 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:07:22.241687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239006 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:07:22.241687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239008 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:07:22.241687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239011 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:07:22.241687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239013 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:07:22.241687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239016 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:07:22.242181 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239018 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:07:22.242181 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239021 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:07:22.242181 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239042 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:07:22.242181 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239045 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:07:22.242181 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239049 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:07:22.242181 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239053 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:07:22.242181 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239056 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:07:22.242181 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239059 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:07:22.242181 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239062 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:07:22.242181 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239064 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:07:22.242181 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239067 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:07:22.242181 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239069 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:07:22.242181 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239072 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:07:22.242181 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239074 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:07:22.242181 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239077 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:07:22.242181 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239081 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:07:22.242181 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239084 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:07:22.242181 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239086 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:07:22.242181 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239090 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:07:22.242669 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239094 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:07:22.242669 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239098 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:07:22.242669 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239101 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:07:22.242669 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239104 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:07:22.242669 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239107 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:07:22.242669 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239111 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:07:22.242669 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239115 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:07:22.242669 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239121 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:07:22.242669 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239125 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:07:22.242669 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239129 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:07:22.242669 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239133 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:07:22.242669 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239136 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:07:22.242669 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239677 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:07:22.242669 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239683 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:07:22.242669 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239687 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:07:22.242669 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239690 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:07:22.242669 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239693 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:07:22.242669 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239695 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:07:22.242669 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239698 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:07:22.243128 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239703 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:07:22.243128 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239706 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:07:22.243128 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239709 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:07:22.243128 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239711 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:07:22.243128 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239714 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:07:22.243128 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239717 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:07:22.243128 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239720 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:07:22.243128 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239723 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:07:22.243128 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239726 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:07:22.243128 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239730 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:07:22.243128 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239733 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:07:22.243128 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239735 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:07:22.243128 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239738 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:07:22.243128 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239741 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:07:22.243128 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239743 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:07:22.243128 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239746 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:07:22.243128 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239748 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:07:22.243128 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239751 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:07:22.243128 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239753 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:07:22.243591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239756 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:07:22.243591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239759 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:07:22.243591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239761 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:07:22.243591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239764 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:07:22.243591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239767 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:07:22.243591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239769 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:07:22.243591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239772 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:07:22.243591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239775 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:07:22.243591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239778 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:07:22.243591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239781 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:07:22.243591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239783 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:07:22.243591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239787 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:07:22.243591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239791 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:07:22.243591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239794 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:07:22.243591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239797 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:07:22.243591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239800 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:07:22.243591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239802 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:07:22.243591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239804 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:07:22.243591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239808 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:07:22.243591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239810 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:07:22.244077 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239813 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:07:22.244077 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239815 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:07:22.244077 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239819 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:07:22.244077 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239821 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:07:22.244077 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239824 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:07:22.244077 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239827 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:07:22.244077 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239829 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:07:22.244077 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239832 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:07:22.244077 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239835 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:07:22.244077 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239837 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:07:22.244077 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239840 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:07:22.244077 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239842 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:07:22.244077 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239844 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:07:22.244077 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239847 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:07:22.244077 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239850 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:07:22.244077 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239852 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:07:22.244077 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239856 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:07:22.244077 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239858 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:07:22.244077 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239861 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:07:22.244077 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239863 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:07:22.244551 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239866 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:07:22.244551 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239868 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:07:22.244551 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239873 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:07:22.244551 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239875 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:07:22.244551 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239878 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:07:22.244551 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239880 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:07:22.244551 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239882 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:07:22.244551 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239885 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:07:22.244551 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239888 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:07:22.244551 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239890 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:07:22.244551 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239894 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:07:22.244551 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239896 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:07:22.244551 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239914 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:07:22.244551 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239918 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:07:22.244551 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239924 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:07:22.244551 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239928 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:07:22.244551 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239932 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:07:22.244551 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239935 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:07:22.244551 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239940 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:07:22.244551 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239943 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:07:22.245038 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239947 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:07:22.245038 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239950 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:07:22.245038 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239954 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:07:22.245038 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239957 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:07:22.245038 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239961 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:07:22.245038 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239966 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:07:22.245038 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239969 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:07:22.245038 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239972 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:07:22.245038 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239975 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:07:22.245038 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239978 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:07:22.245038 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239980 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:07:22.245038 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239983 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:07:22.245038 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239986 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:07:22.245038 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239988 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:07:22.245038 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.239991 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:07:22.245038 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.240000 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:07:22.245038 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.240004 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:07:22.245038 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.240007 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:07:22.245038 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.240009 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.240012 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240820 2579 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240829 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240837 2579 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240841 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240846 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240849 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240854 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240859 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240862 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240865 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240869 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240872 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240875 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240878 2579 flags.go:64] FLAG: --cgroup-root="" Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240881 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240884 2579 flags.go:64] FLAG: --client-ca-file="" Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240887 2579 flags.go:64] FLAG: --cloud-provider="external" Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240890 2579 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240896 2579 flags.go:64] FLAG: --cluster-domain="" Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240899 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240902 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240905 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 22:07:22.245498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240908 2579 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240912 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240915 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240919 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240922 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240925 2579 flags.go:64] FLAG: --contention-profiling="false" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240929 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240932 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240935 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240938 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240943 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240946 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240951 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240954 2579 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240957 2579 flags.go:64] FLAG: --enable-server="true" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240959 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240964 2579 flags.go:64] FLAG: --event-burst="100" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240968 2579 flags.go:64] FLAG: --event-qps="50" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240971 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240974 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240977 2579 flags.go:64] FLAG: --eviction-hard="" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240981 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240985 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240988 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240991 2579 flags.go:64] FLAG: --eviction-soft="" Oct 30 22:07:22.246081 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240994 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.240997 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241000 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241003 2579 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241006 2579 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241009 2579 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241012 2579 flags.go:64] FLAG: --feature-gates="" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241016 2579 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241019 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241037 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241042 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241047 2579 flags.go:64] FLAG: --healthz-port="10248" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241051 2579 flags.go:64] FLAG: --help="false" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241056 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-131-94.ec2.internal" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241061 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241066 2579 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241069 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241073 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241076 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241081 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241084 2579 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241087 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241090 2579 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 22:07:22.246738 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241092 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241096 2579 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241098 2579 flags.go:64] FLAG: --kube-reserved="" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241101 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241105 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241108 2579 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241110 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241113 2579 flags.go:64] FLAG: --lock-file="" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241116 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241119 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241123 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241129 2579 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241132 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241135 2579 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241138 2579 flags.go:64] FLAG: --logging-format="text" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241140 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241144 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241146 2579 flags.go:64] FLAG: --manifest-url="" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241149 2579 flags.go:64] FLAG: --manifest-url-header="" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241154 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241157 2579 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241161 2579 flags.go:64] FLAG: --max-pods="110" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241164 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241167 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241170 2579 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 22:07:22.247415 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241174 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241177 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241180 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241184 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241191 2579 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241194 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241198 2579 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241201 2579 flags.go:64] FLAG: --pod-cidr="" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241204 2579 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241207 2579 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241210 2579 flags.go:64] FLAG: --pods-per-core="0" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241213 2579 flags.go:64] FLAG: --port="10250" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241216 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241219 2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-060ca0ed5af39e278" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241222 2579 flags.go:64] FLAG: --qos-reserved="" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241225 2579 flags.go:64] FLAG: --read-only-port="10255" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241228 2579 flags.go:64] FLAG: --register-node="true" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241231 2579 flags.go:64] FLAG: --register-with-taints="" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241235 2579 flags.go:64] FLAG: --registry-burst="10" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241237 2579 flags.go:64] FLAG: --registry-qps="5" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241240 2579 flags.go:64] FLAG: --reserved-cpus="" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241243 2579 flags.go:64] FLAG: --reserved-memory="" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241247 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241250 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241253 2579 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241255 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 22:07:22.248077 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241258 2579 flags.go:64] FLAG: --runonce="false" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241261 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241264 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241267 2579 flags.go:64] FLAG: --seccomp-default="false" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241270 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241273 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241286 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241289 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241292 2579 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241297 2579 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241302 2579 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241306 2579 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241309 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241312 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241315 2579 flags.go:64] FLAG: --system-cgroups="" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241318 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241324 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241326 2579 flags.go:64] FLAG: --tls-cert-file="" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241329 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241334 2579 flags.go:64] FLAG: --tls-min-version="" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241337 2579 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241340 2579 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241343 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241346 2579 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241349 2579 flags.go:64] FLAG: --v="2" Oct 30 22:07:22.248702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241358 2579 flags.go:64] FLAG: --version="false" Oct 30 22:07:22.249312 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241362 2579 flags.go:64] FLAG: --vmodule="" Oct 30 22:07:22.249312 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241366 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 22:07:22.249312 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241370 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 22:07:22.249312 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241508 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:07:22.249312 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241512 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:07:22.249312 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241516 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:07:22.249312 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241519 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:07:22.249312 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241521 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:07:22.249312 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241524 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:07:22.249312 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241526 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:07:22.249312 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241529 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:07:22.249312 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241532 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:07:22.249312 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241535 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:07:22.249312 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241538 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:07:22.249312 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241540 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:07:22.249312 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241544 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:07:22.249312 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241550 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:07:22.249312 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241554 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:07:22.249312 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241557 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:07:22.249312 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241559 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:07:22.249803 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241562 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:07:22.249803 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241565 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:07:22.249803 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241568 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:07:22.249803 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241570 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:07:22.249803 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241573 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:07:22.249803 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241575 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:07:22.249803 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241578 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:07:22.249803 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241581 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:07:22.249803 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241584 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:07:22.249803 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241586 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:07:22.249803 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241589 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:07:22.249803 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241591 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:07:22.249803 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241594 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:07:22.249803 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241596 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:07:22.249803 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241599 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:07:22.249803 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241601 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:07:22.249803 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241604 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:07:22.249803 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241607 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:07:22.249803 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241609 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:07:22.249803 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241613 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:07:22.250300 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241615 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:07:22.250300 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241618 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:07:22.250300 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241621 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:07:22.250300 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241623 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:07:22.250300 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241626 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:07:22.250300 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241628 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:07:22.250300 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241631 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:07:22.250300 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241636 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:07:22.250300 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241641 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:07:22.250300 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241646 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:07:22.250300 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241649 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:07:22.250300 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241652 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:07:22.250300 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241655 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:07:22.250300 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241658 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:07:22.250300 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241660 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:07:22.250300 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241663 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:07:22.250300 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241665 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:07:22.250300 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241668 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:07:22.250300 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241670 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:07:22.250764 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241682 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:07:22.250764 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241685 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:07:22.250764 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241688 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:07:22.250764 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241691 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:07:22.250764 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241694 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:07:22.250764 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241697 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:07:22.250764 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241700 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:07:22.250764 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241703 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:07:22.250764 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241706 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:07:22.250764 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241709 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:07:22.250764 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241711 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:07:22.250764 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241714 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:07:22.250764 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241716 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:07:22.250764 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241719 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:07:22.250764 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241722 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:07:22.250764 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241724 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:07:22.250764 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241726 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:07:22.250764 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241729 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:07:22.250764 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241731 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:07:22.251222 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241734 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:07:22.251222 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241738 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:07:22.251222 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241742 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:07:22.251222 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241745 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:07:22.251222 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241747 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:07:22.251222 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241750 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:07:22.251222 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241752 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:07:22.251222 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241755 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:07:22.251222 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241758 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:07:22.251222 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241760 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:07:22.251222 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241763 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:07:22.251222 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241766 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:07:22.251222 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241768 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:07:22.251222 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241771 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:07:22.251222 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241773 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:07:22.251222 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241776 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:07:22.251222 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241779 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:07:22.251222 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241781 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:07:22.251222 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241784 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:07:22.251718 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241786 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:07:22.251718 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241789 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:07:22.251718 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241792 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:07:22.251718 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241795 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:07:22.251718 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241797 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:07:22.251718 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241800 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:07:22.251718 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241802 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:07:22.251718 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241805 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:07:22.251718 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241808 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:07:22.251718 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241810 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:07:22.251718 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241813 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:07:22.251718 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241815 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:07:22.251718 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.241842 2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 22:07:22.251718 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241878 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:07:22.251718 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241882 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:07:22.251718 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241885 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:07:22.252126 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241890 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:07:22.252126 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241893 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:07:22.252126 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241896 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:07:22.252126 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241898 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:07:22.252126 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241901 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:07:22.252126 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241904 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:07:22.252126 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241906 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:07:22.252126 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241909 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:07:22.252126 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241911 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:07:22.252126 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241914 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:07:22.252126 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241917 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:07:22.252126 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241919 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:07:22.252126 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241922 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:07:22.252126 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241925 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:07:22.252126 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241928 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:07:22.252126 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241932 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:07:22.252126 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241935 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:07:22.252126 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241939 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:07:22.252126 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241942 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:07:22.252590 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241945 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:07:22.252590 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241948 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:07:22.252590 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241950 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:07:22.252590 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241953 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:07:22.252590 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241955 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:07:22.252590 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241958 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:07:22.252590 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241960 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:07:22.252590 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241963 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:07:22.252590 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241966 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:07:22.252590 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241968 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:07:22.252590 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241971 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:07:22.252590 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241974 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:07:22.252590 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241977 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:07:22.252590 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241982 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:07:22.252590 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241985 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:07:22.252590 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241987 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:07:22.252590 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241990 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:07:22.252590 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241993 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:07:22.252590 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241995 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:07:22.253067 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.241998 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:07:22.253067 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242000 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:07:22.253067 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242003 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:07:22.253067 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242005 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:07:22.253067 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242008 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:07:22.253067 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242010 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:07:22.253067 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242013 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:07:22.253067 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242015 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:07:22.253067 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242018 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:07:22.253067 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242021 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:07:22.253067 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242035 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:07:22.253067 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242038 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:07:22.253067 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242041 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:07:22.253067 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242043 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:07:22.253067 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242046 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:07:22.253067 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242049 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:07:22.253067 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242052 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:07:22.253067 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242054 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:07:22.253067 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242057 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:07:22.253518 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242059 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:07:22.253518 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242062 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:07:22.253518 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242066 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:07:22.253518 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242068 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:07:22.253518 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242071 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:07:22.253518 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242074 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:07:22.253518 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242077 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:07:22.253518 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242081 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:07:22.253518 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242084 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:07:22.253518 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242086 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:07:22.253518 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242089 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:07:22.253518 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242091 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:07:22.253518 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242094 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:07:22.253518 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242097 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:07:22.253518 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242099 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:07:22.253518 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242102 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:07:22.253518 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242104 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:07:22.253518 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242106 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:07:22.253518 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242109 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:07:22.253518 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242112 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:07:22.254005 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242115 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:07:22.254005 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242117 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:07:22.254005 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242120 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:07:22.254005 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242123 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:07:22.254005 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242125 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:07:22.254005 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242128 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:07:22.254005 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242130 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:07:22.254005 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242133 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:07:22.254005 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242136 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:07:22.254005 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242138 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:07:22.254005 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242141 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:07:22.254005 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242144 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:07:22.254005 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242148 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:07:22.254005 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242151 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:07:22.254005 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242153 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:07:22.254005 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242156 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:07:22.254005 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242159 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:07:22.254005 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242162 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:07:22.254005 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242165 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:07:22.254476 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242169 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:07:22.254476 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242171 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:07:22.254476 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242174 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:07:22.254476 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242178 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:07:22.254476 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242181 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:07:22.254476 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242184 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:07:22.254476 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.242186 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:07:22.254476 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.248604 2579 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 22:07:22.254476 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.248622 2579 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 22:07:22.254476 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.249175 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 22:07:22.254476 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.249183 2579 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 22:07:22.254476 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249243 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:07:22.254476 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249249 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:07:22.254476 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249253 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:07:22.254476 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249256 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:07:22.254476 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249259 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:07:22.254476 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249262 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:07:22.254476 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249265 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:07:22.254476 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249268 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:07:22.254476 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249271 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:07:22.254976 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249274 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:07:22.254976 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249276 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:07:22.254976 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249279 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:07:22.254976 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249282 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:07:22.254976 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249284 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:07:22.254976 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249287 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:07:22.254976 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249290 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:07:22.254976 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249293 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:07:22.254976 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249295 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:07:22.254976 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249298 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:07:22.254976 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249300 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:07:22.254976 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249303 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:07:22.254976 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249306 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:07:22.254976 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249308 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:07:22.254976 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249311 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:07:22.254976 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249314 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:07:22.254976 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249317 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:07:22.254976 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249319 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:07:22.254976 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249322 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:07:22.254976 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249325 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:07:22.255486 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249330 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:07:22.255486 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249333 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:07:22.255486 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249335 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:07:22.255486 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249338 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:07:22.255486 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249341 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:07:22.255486 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249344 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:07:22.255486 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249346 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:07:22.255486 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249349 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:07:22.255486 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249351 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:07:22.255486 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249354 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:07:22.255486 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249356 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:07:22.255486 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249359 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:07:22.255486 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249362 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:07:22.255486 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249364 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:07:22.255486 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249367 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:07:22.255486 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249369 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:07:22.255486 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249372 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:07:22.255486 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249375 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:07:22.255486 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249379 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:07:22.255933 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249383 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:07:22.255933 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249386 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:07:22.255933 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249389 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:07:22.255933 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249392 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:07:22.255933 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249395 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:07:22.255933 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249398 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:07:22.255933 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249400 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:07:22.255933 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249403 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:07:22.255933 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249406 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:07:22.255933 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249409 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:07:22.255933 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249412 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:07:22.255933 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249414 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:07:22.255933 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249417 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:07:22.255933 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249420 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:07:22.255933 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249432 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:07:22.255933 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249435 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:07:22.255933 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249438 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:07:22.255933 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249440 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:07:22.255933 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249443 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:07:22.255933 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249446 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:07:22.256434 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249448 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:07:22.256434 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249451 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:07:22.256434 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249453 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:07:22.256434 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249456 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:07:22.256434 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249458 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:07:22.256434 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249478 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:07:22.256434 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249481 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:07:22.256434 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249484 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:07:22.256434 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249488 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:07:22.256434 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249491 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:07:22.256434 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249494 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:07:22.256434 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249497 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:07:22.256434 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249500 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:07:22.256434 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249502 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:07:22.256434 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249505 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:07:22.256434 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249508 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:07:22.256434 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249510 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:07:22.256434 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249514 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:07:22.256434 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249518 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:07:22.256949 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249521 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:07:22.256949 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249524 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:07:22.256949 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249527 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:07:22.256949 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249530 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:07:22.256949 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249532 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:07:22.256949 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249535 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:07:22.256949 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249538 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:07:22.256949 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249541 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:07:22.256949 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249557 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:07:22.256949 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249561 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:07:22.256949 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249564 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:07:22.256949 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249566 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:07:22.256949 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249569 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:07:22.256949 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249572 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:07:22.256949 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249574 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:07:22.256949 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249577 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:07:22.256949 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249579 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:07:22.256949 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249582 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:07:22.256949 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249584 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:07:22.257795 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.249614 2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 22:07:22.257795 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249804 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:07:22.257795 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249809 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:07:22.257795 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249813 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:07:22.257795 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249816 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:07:22.257795 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249819 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:07:22.257795 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249822 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:07:22.257795 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249824 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:07:22.257795 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249827 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:07:22.257795 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249834 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:07:22.257795 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249837 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:07:22.257795 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249840 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:07:22.257795 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249842 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:07:22.257795 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249845 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:07:22.257795 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249847 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:07:22.257795 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249850 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:07:22.258444 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249853 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:07:22.258444 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249855 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:07:22.258444 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249858 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:07:22.258444 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249860 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:07:22.258444 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249863 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:07:22.258444 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249865 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:07:22.258444 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249874 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:07:22.258444 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249877 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:07:22.258444 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249879 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:07:22.258444 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249882 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:07:22.258444 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249884 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:07:22.258444 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249887 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:07:22.258444 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249890 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:07:22.258444 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249892 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:07:22.258444 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249894 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:07:22.258444 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249897 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:07:22.258444 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249899 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:07:22.258444 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249902 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:07:22.258444 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249904 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:07:22.258444 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249907 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:07:22.259108 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249909 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:07:22.259108 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249912 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:07:22.259108 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249914 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:07:22.259108 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249917 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:07:22.259108 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249919 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:07:22.259108 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249922 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:07:22.259108 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249924 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:07:22.259108 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249927 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:07:22.259108 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249929 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:07:22.259108 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249932 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:07:22.259108 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249934 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:07:22.259108 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249937 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:07:22.259108 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249939 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:07:22.259108 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249942 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:07:22.259108 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249944 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:07:22.259108 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249947 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:07:22.259108 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249950 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:07:22.259108 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249952 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:07:22.259108 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249955 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:07:22.259697 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249963 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:07:22.259697 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249966 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:07:22.259697 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249968 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:07:22.259697 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249971 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:07:22.259697 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249973 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:07:22.259697 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249976 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:07:22.259697 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249979 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:07:22.259697 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249981 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:07:22.259697 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249983 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:07:22.259697 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249986 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:07:22.259697 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249988 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:07:22.259697 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249991 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:07:22.259697 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249994 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:07:22.259697 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249996 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:07:22.259697 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.249999 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:07:22.259697 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250001 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:07:22.259697 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250004 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:07:22.259697 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250006 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:07:22.259697 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250009 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:07:22.259697 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250011 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:07:22.260275 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250014 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:07:22.260275 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250037 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:07:22.260275 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250044 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:07:22.260275 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250047 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:07:22.260275 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250050 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:07:22.260275 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250053 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:07:22.260275 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250056 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:07:22.260275 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250058 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:07:22.260275 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250061 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:07:22.260275 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250064 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:07:22.260275 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250066 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:07:22.260275 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250069 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:07:22.260275 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250072 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:07:22.260275 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250081 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:07:22.260275 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250083 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:07:22.260275 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250087 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:07:22.260275 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250091 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:07:22.260275 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250094 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:07:22.260275 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250097 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:07:22.260752 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250100 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:07:22.260752 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250102 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:07:22.260752 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250105 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:07:22.260752 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250107 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:07:22.260752 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250110 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:07:22.260752 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250112 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:07:22.260752 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250115 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:07:22.260752 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250117 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:07:22.260752 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250120 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:07:22.260752 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250122 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:07:22.260752 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250125 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:07:22.260752 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250127 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:07:22.260752 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250130 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:07:22.260752 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.250154 2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 22:07:22.260752 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250193 2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:07:22.260752 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250198 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:07:22.261163 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250200 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:07:22.261163 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250204 2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:07:22.261163 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250206 2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:07:22.261163 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250209 2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:07:22.261163 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250211 2579 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:07:22.261163 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250214 2579 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:07:22.261163 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250219 2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:07:22.261163 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250222 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:07:22.261163 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250224 2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:07:22.261163 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250227 2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:07:22.261163 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250229 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:07:22.261163 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250238 2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:07:22.261163 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250241 2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:07:22.261163 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250243 2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:07:22.261163 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250246 2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:07:22.261163 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250249 2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:07:22.261163 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250252 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:07:22.261163 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250254 2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:07:22.261163 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250257 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:07:22.261163 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250259 2579 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:07:22.261687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250262 2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:07:22.261687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250265 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:07:22.261687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250267 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:07:22.261687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250270 2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:07:22.261687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250273 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:07:22.261687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250275 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:07:22.261687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250278 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:07:22.261687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250280 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:07:22.261687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250283 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:07:22.261687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250285 2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:07:22.261687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250288 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:07:22.261687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250290 2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:07:22.261687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250293 2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:07:22.261687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250296 2579 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:07:22.261687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250298 2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:07:22.261687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250300 2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:07:22.261687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250303 2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:07:22.261687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250306 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:07:22.261687 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250310 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:07:22.262150 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250313 2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:07:22.262150 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250315 2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:07:22.262150 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250318 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:07:22.262150 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250320 2579 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:07:22.262150 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250323 2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:07:22.262150 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250332 2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:07:22.262150 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250336 2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:07:22.262150 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250339 2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:07:22.262150 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250343 2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:07:22.262150 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250347 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:07:22.262150 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250350 2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:07:22.262150 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250352 2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:07:22.262150 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250355 2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:07:22.262150 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250358 2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:07:22.262150 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250361 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:07:22.262150 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250364 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:07:22.262150 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250366 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:07:22.262150 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250369 2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:07:22.262150 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250371 2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:07:22.262591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250374 2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:07:22.262591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250376 2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:07:22.262591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250381 2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:07:22.262591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250384 2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:07:22.262591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250387 2579 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:07:22.262591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250389 2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:07:22.262591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250392 2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:07:22.262591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250395 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:07:22.262591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250397 2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:07:22.262591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250400 2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:07:22.262591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250402 2579 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:07:22.262591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250405 2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:07:22.262591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250408 2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:07:22.262591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250410 2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:07:22.262591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250413 2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:07:22.262591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250415 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:07:22.262591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250418 2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:07:22.262591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250420 2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:07:22.262591 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250423 2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:07:22.263048 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250433 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:07:22.263048 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250435 2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:07:22.263048 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250438 2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:07:22.263048 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250440 2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:07:22.263048 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250443 2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:07:22.263048 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250445 2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:07:22.263048 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250448 2579 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:07:22.263048 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250451 2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:07:22.263048 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250454 2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:07:22.263048 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250456 2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:07:22.263048 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250459 2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:07:22.263048 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250461 2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:07:22.263048 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250464 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:07:22.263048 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250466 2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:07:22.263048 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250469 2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:07:22.263048 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250471 2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:07:22.263048 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250475 2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:07:22.263048 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250477 2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:07:22.263048 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250480 2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:07:22.263048 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250482 2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:07:22.263583 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250485 2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:07:22.263583 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250487 2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:07:22.263583 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250490 2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:07:22.263583 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250492 2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:07:22.263583 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250494 2579 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:07:22.263583 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250497 2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:07:22.263583 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:22.250500 2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:07:22.263583 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.250560 2579 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 22:07:22.263583 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.252967 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Oct 30 22:07:22.263583 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.255251 2579 server.go:1014] "Starting client certificate rotation" Oct 30 22:07:22.263583 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.255389 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 22:07:22.263583 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.255435 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 22:07:22.263583 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.259584 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 22:07:22.263583 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.260731 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 22:07:22.270614 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.270597 2579 log.go:25] "Validated CRI v1 runtime API" Oct 30 22:07:22.273541 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.273521 2579 log.go:25] "Validated CRI v1 image API" Oct 30 22:07:22.274508 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.274481 2579 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 22:07:22.279381 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.279353 2579 fs.go:135] Filesystem UUIDs: map[4b5c7890-f881-45d1-ae2c-18e754c8b3dd:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 fef1b876-f40d-4e35-9a1c-db84555c057a:/dev/nvme0n1p4] Oct 30 22:07:22.279434 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.279380 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Oct 30 22:07:22.285149 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.285019 2579 manager.go:217] Machine: {Timestamp:2026-10-30 22:07:22.283898792 +0000 UTC m=+0.314989535 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3092731 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bafde1fd402ce4526e077dca30f75 SystemUUID:ec2bafde-1fd4-02ce-4526-e077dca30f75 BootID:458000f1-c64c-476b-b69d-b4ec70cc87b2 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:db:d1:4a:ef Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:db:d1:4a:ef Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:92:fb:dd:1e:95 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 22:07:22.285252 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.285152 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 22:07:22.285282 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.285247 2579 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 22:07:22.286266 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.286238 2579 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 22:07:22.286424 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.286267 2579 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-94.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 22:07:22.286466 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.286425 2579 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 22:07:22.286466 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.286433 2579 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 22:07:22.286529 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.286468 2579 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 22:07:22.286529 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.286487 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 22:07:22.287258 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.287248 2579 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 22:07:22.291275 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.291263 2579 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 22:07:22.291403 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.291393 2579 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 22:07:22.292340 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.292330 2579 kubelet.go:493] "Attempting to sync node with API server" Oct 30 22:07:22.292426 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.292343 2579 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 22:07:22.292426 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.292357 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 22:07:22.292426 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.292366 2579 kubelet.go:405] "Adding apiserver pod source" Oct 30 22:07:22.292426 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.292389 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 22:07:22.292854 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.292838 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 22:07:22.296056 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.296013 2579 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 22:07:22.300659 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.300642 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 22:07:22.301336 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.301324 2579 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 22:07:22.301372 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.301355 2579 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 22:07:22.301515 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.301505 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 22:07:22.301543 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.301522 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 22:07:22.301543 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.301528 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 22:07:22.301543 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.301534 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 22:07:22.301543 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.301540 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 22:07:22.301648 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.301549 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 22:07:22.302052 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.302043 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 22:07:22.302087 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.302053 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 22:07:22.302087 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.302064 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 22:07:22.302087 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.302073 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 22:07:22.302087 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.302082 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 22:07:22.302253 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.302091 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 22:07:22.302718 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.302707 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 22:07:22.302757 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.302720 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 22:07:22.306546 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.306532 2579 server.go:1263] "Started kubelet" Oct 30 22:07:22.306664 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.306634 2579 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 22:07:22.306719 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.306625 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 22:07:22.306775 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.306719 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 22:07:22.307346 ip-10-0-131-94 systemd[1]: Started Kubernetes Kubelet. Oct 30 22:07:22.311043 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.309167 2579 server.go:317] "Adding debug handlers to kubelet server" Oct 30 22:07:22.311043 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.309396 2579 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 22:07:22.313825 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.313807 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 22:07:22.314197 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.314178 2579 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 22:07:22.314281 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:22.314255 2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-94.ec2.internal\" not found" Oct 30 22:07:22.314281 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:22.314255 2579 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Oct 30 22:07:22.315067 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.315052 2579 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 22:07:22.315160 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.315072 2579 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 22:07:22.315160 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.315053 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 22:07:22.315160 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.315147 2579 factory.go:55] Registering systemd factory Oct 30 22:07:22.315284 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.315163 2579 factory.go:223] Registration of the systemd container factory successfully Oct 30 22:07:22.315284 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.315195 2579 reconstruct.go:97] "Volume reconstruction finished" Oct 30 22:07:22.315284 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.315204 2579 reconciler.go:29] "Reconciler: start to sync state" Oct 30 22:07:22.315420 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.315402 2579 factory.go:153] Registering CRI-O factory Oct 30 22:07:22.315466 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.315423 2579 factory.go:223] Registration of the crio container factory successfully Oct 30 22:07:22.315495 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.315484 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 22:07:22.315525 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.315504 2579 factory.go:103] Registering Raw factory Oct 30 22:07:22.315525 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.315520 2579 manager.go:1199] Started watching for new ooms in manager Oct 30 22:07:22.316364 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.316342 2579 manager.go:319] Starting recovery of all containers Oct 30 22:07:22.324939 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.324884 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 22:07:22.328413 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.328396 2579 manager.go:324] Recovery completed Oct 30 22:07:22.334550 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.333968 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 22:07:22.335854 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.335833 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientMemory" Oct 30 22:07:22.335935 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.335864 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 22:07:22.335935 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.335874 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientPID" Oct 30 22:07:22.336462 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.336448 2579 cpu_manager.go:226] "Starting" policy="none" Oct 30 22:07:22.336516 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.336462 2579 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 22:07:22.336516 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.336450 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 22:07:22.336516 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.336484 2579 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 22:07:22.337170 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.337155 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientMemory" Oct 30 22:07:22.337224 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.337182 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 22:07:22.337224 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.337218 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientPID" Oct 30 22:07:22.338768 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.338754 2579 policy_none.go:50] "Start" Oct 30 22:07:22.338818 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.338774 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 22:07:22.338818 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.338802 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 22:07:22.340607 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:22.339776 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-94.ec2.internal.18e36e04848f6da4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-94.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-94.ec2.internal,},FirstTimestamp:2026-10-30 22:07:22.306506148 +0000 UTC m=+0.337596887,LastTimestamp:2026-10-30 22:07:22.306506148 +0000 UTC m=+0.337596887,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-94.ec2.internal,}" Oct 30 22:07:22.340827 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.340815 2579 policy_none.go:44] "Start" Oct 30 22:07:22.348108 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.348090 2579 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 22:07:22.348179 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.348155 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7j8xr" Oct 30 22:07:22.351138 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.351119 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7j8xr" Oct 30 22:07:22.351289 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.351270 2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-94.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Oct 30 22:07:22.351350 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:22.351322 2579 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-94.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Oct 30 22:07:22.356236 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:22.355364 2579 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" Oct 30 22:07:22.356236 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:22.355415 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-94.ec2.internal.18e36e04864f3352 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-94.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-94.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-94.ec2.internal,},FirstTimestamp:2026-10-30 22:07:22.335851346 +0000 UTC m=+0.366942085,LastTimestamp:2026-10-30 22:07:22.335851346 +0000 UTC m=+0.366942085,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-94.ec2.internal,}" Oct 30 22:07:22.356236 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:22.355503 2579 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" Oct 30 22:07:22.371661 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.371620 2579 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 22:07:22.376056 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.376038 2579 manager.go:341] "Starting Device Plugin manager" Oct 30 22:07:22.376140 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:22.376073 2579 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 22:07:22.376140 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.376086 2579 server.go:87] "Starting device plugin registration server" Oct 30 22:07:22.376352 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.376338 2579 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 22:07:22.376418 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.376352 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 22:07:22.376498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.376481 2579 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 22:07:22.376582 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.376563 2579 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 22:07:22.376582 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.376575 2579 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 22:07:22.377128 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:22.377103 2579 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 22:07:22.377220 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:22.377154 2579 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-94.ec2.internal\" not found" Oct 30 22:07:22.379771 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.379753 2579 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 22:07:22.457196 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.457173 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 22:07:22.457196 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.457200 2579 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 22:07:22.457384 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.457218 2579 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 22:07:22.457384 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:22.457252 2579 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Oct 30 22:07:22.470810 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.470788 2579 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 22:07:22.476877 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.476862 2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 22:07:22.477811 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.477794 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientMemory" Oct 30 22:07:22.477901 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.477822 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 22:07:22.477901 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.477832 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientPID" Oct 30 22:07:22.477901 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.477857 2579 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-131-94.ec2.internal" Oct 30 22:07:22.495806 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.495757 2579 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-131-94.ec2.internal" Oct 30 22:07:22.558366 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.558324 2579 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal"] Oct 30 22:07:22.560728 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.560711 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Oct 30 22:07:22.562510 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.562494 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" Oct 30 22:07:22.594829 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.594812 2579 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Oct 30 22:07:22.597478 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.597463 2579 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" Oct 30 22:07:22.616925 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.616891 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/00a20884b67d7fdeb7bbc7e471370a8a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal\" (UID: \"00a20884b67d7fdeb7bbc7e471370a8a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Oct 30 22:07:22.616925 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.616928 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00a20884b67d7fdeb7bbc7e471370a8a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal\" (UID: \"00a20884b67d7fdeb7bbc7e471370a8a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Oct 30 22:07:22.617083 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.616950 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f39ce36b79a5ed24b44c91f0e6e003f7-config\") pod \"kube-apiserver-proxy-ip-10-0-131-94.ec2.internal\" (UID: \"f39ce36b79a5ed24b44c91f0e6e003f7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" Oct 30 22:07:22.617606 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.617588 2579 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 22:07:22.617766 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.617750 2579 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 22:07:22.718153 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.718090 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00a20884b67d7fdeb7bbc7e471370a8a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal\" (UID: \"00a20884b67d7fdeb7bbc7e471370a8a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Oct 30 22:07:22.718153 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.718147 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f39ce36b79a5ed24b44c91f0e6e003f7-config\") pod \"kube-apiserver-proxy-ip-10-0-131-94.ec2.internal\" (UID: \"f39ce36b79a5ed24b44c91f0e6e003f7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" Oct 30 22:07:22.718153 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.718164 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/00a20884b67d7fdeb7bbc7e471370a8a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal\" (UID: \"00a20884b67d7fdeb7bbc7e471370a8a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Oct 30 22:07:22.718395 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.718190 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/00a20884b67d7fdeb7bbc7e471370a8a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal\" (UID: \"00a20884b67d7fdeb7bbc7e471370a8a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Oct 30 22:07:22.718395 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.718193 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00a20884b67d7fdeb7bbc7e471370a8a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal\" (UID: \"00a20884b67d7fdeb7bbc7e471370a8a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Oct 30 22:07:22.718395 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.718197 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f39ce36b79a5ed24b44c91f0e6e003f7-config\") pod \"kube-apiserver-proxy-ip-10-0-131-94.ec2.internal\" (UID: \"f39ce36b79a5ed24b44c91f0e6e003f7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" Oct 30 22:07:22.922528 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.922439 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" Oct 30 22:07:22.922672 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:22.922450 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Oct 30 22:07:24.793373 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.793156 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Oct 30 22:07:24.793818 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.793485 2579 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Oct 30 22:07:24.831513 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.831484 2579 apiserver.go:52] "Watching apiserver" Oct 30 22:07:24.853002 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.852977 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Oct 30 22:07:24.874295 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.874263 2579 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 22:07:24.874564 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.874543 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal","openshift-multus/network-metrics-daemon-l7k26","openshift-image-registry/node-ca-h9qqf","openshift-network-diagnostics/network-check-target-pmkb2","openshift-network-operator/iptables-alerter-zt9w6","openshift-ovn-kubernetes/ovnkube-node-wqj4x","kube-system/global-pull-secret-syncer-gsvm9","kube-system/konnectivity-agent-6qz5h","kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal","openshift-multus/multus-4gdsr","openshift-multus/multus-additional-cni-plugins-lzh8c","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79","openshift-cluster-node-tuning-operator/tuned-w692c","openshift-dns/node-resolver-q5gw7"] Oct 30 22:07:24.877023 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.877006 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:24.877103 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:24.877086 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7k26" podUID="f805f34e-7fc9-430b-9003-16d9765d6657" Oct 30 22:07:24.879016 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.879001 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h9qqf" Oct 30 22:07:24.881314 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.881295 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:24.881387 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:24.881360 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmkb2" podUID="d3d3d52b-f209-4ad4-b531-4fe19bad4bf1" Oct 30 22:07:24.883425 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.883413 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zt9w6" Oct 30 22:07:24.885580 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.885562 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:24.887703 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.887682 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:24.887772 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:24.887750 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsvm9" podUID="bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9" Oct 30 22:07:24.889837 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.889816 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6qz5h" Oct 30 22:07:24.892017 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.892001 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4gdsr" Oct 30 22:07:24.893748 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.893724 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 22:02:23 +0000 UTC" deadline="2028-07-26 04:38:56.37579925 +0000 UTC" Oct 30 22:07:24.893748 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.893746 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15222h31m31.482055425s" Oct 30 22:07:24.894000 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.893990 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pcvff\"" Oct 30 22:07:24.894222 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.894205 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 22:07:24.894297 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.894284 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 22:07:24.894334 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.894300 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 22:07:24.894334 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.894321 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:24.896432 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.896419 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:24.897019 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.897004 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 22:07:24.898584 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.898569 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:24.899207 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.899191 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 22:07:24.899831 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.899817 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 22:07:24.899923 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.899835 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 22:07:24.899923 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.899875 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 22:07:24.899923 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.899875 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 22:07:24.899923 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.899894 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 22:07:24.899923 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.899885 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-khd2p\"" Oct 30 22:07:24.899923 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.899918 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 22:07:24.900164 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.899955 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 22:07:24.900164 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.900007 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4vl25\"" Oct 30 22:07:24.900884 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.900871 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q5gw7" Oct 30 22:07:24.905378 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.905362 2579 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 22:07:24.905446 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.905410 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2m6d2" Oct 30 22:07:24.954767 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.954557 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 22:07:24.958353 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.958330 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-host\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:24.958466 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.958360 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-run-netns\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:24.958466 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.958376 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c135a307-61bc-4e34-9b5c-b97793255db7-tmp-dir\") pod \"node-resolver-q5gw7\" (UID: \"c135a307-61bc-4e34-9b5c-b97793255db7\") " pod="openshift-dns/node-resolver-q5gw7" Oct 30 22:07:24.958466 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.958396 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z8f9\" (UniqueName: \"kubernetes.io/projected/c135a307-61bc-4e34-9b5c-b97793255db7-kube-api-access-8z8f9\") pod \"node-resolver-q5gw7\" (UID: \"c135a307-61bc-4e34-9b5c-b97793255db7\") " pod="openshift-dns/node-resolver-q5gw7" Oct 30 22:07:24.958466 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.958418 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-var-lib-kubelet\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:24.958652 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.958474 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-run-ovn\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:24.958652 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.958524 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-log-socket\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:24.958652 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.958553 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-socket-dir-parent\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:24.958652 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.958594 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-system-cni-dir\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:24.958652 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.958629 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-socket-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:24.958898 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.958674 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wrvl\" (UniqueName: \"kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl\") pod \"network-check-target-pmkb2\" (UID: \"d3d3d52b-f209-4ad4-b531-4fe19bad4bf1\") " pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:24.958898 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.958706 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-var-lib-openvswitch\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:24.958898 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.958744 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt5mm\" (UniqueName: \"kubernetes.io/projected/95637308-9cdc-4a32-b573-70e1b4cff546-kube-api-access-xt5mm\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:24.958898 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.958780 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-hostroot\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:24.958898 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.958809 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-registration-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:24.958898 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.958846 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-run-netns\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:24.958898 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.958874 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-run-openvswitch\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:24.959173 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.958904 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:24.959173 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.958937 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/08904f35-ebda-4220-99c4-b2fb90aa2528-iptables-alerter-script\") pod \"iptables-alerter-zt9w6\" (UID: \"08904f35-ebda-4220-99c4-b2fb90aa2528\") " pod="openshift-network-operator/iptables-alerter-zt9w6" Oct 30 22:07:24.959173 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.958955 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/95637308-9cdc-4a32-b573-70e1b4cff546-ovnkube-script-lib\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:24.959173 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.958969 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-modprobe-d\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:24.959173 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959019 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-sysctl-d\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:24.959173 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959035 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-systemd\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:24.959173 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959118 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-run-k8s-cni-cncf-io\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:24.959173 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959149 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/08904f35-ebda-4220-99c4-b2fb90aa2528-host-slash\") pod \"iptables-alerter-zt9w6\" (UID: \"08904f35-ebda-4220-99c4-b2fb90aa2528\") " pod="openshift-network-operator/iptables-alerter-zt9w6" Oct 30 22:07:24.959173 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959170 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-etc-kubernetes\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:24.959516 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959198 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-etc-selinux\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:24.959516 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959224 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2kvz\" (UniqueName: \"kubernetes.io/projected/08904f35-ebda-4220-99c4-b2fb90aa2528-kube-api-access-m2kvz\") pod \"iptables-alerter-zt9w6\" (UID: \"08904f35-ebda-4220-99c4-b2fb90aa2528\") " pod="openshift-network-operator/iptables-alerter-zt9w6" Oct 30 22:07:24.959516 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959244 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b69c485c-88c0-43c5-a69c-d88c0f81fa22-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:24.959516 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959275 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/95637308-9cdc-4a32-b573-70e1b4cff546-ovn-node-metrics-cert\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:24.959516 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959308 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/12b2045b-a302-47a1-b039-b8eadcc50c61-konnectivity-ca\") pod \"konnectivity-agent-6qz5h\" (UID: \"12b2045b-a302-47a1-b039-b8eadcc50c61\") " pod="kube-system/konnectivity-agent-6qz5h" Oct 30 22:07:24.959516 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959339 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-cni-binary-copy\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:24.959516 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959364 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-daemon-config\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:24.959516 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959380 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffjdk\" (UniqueName: \"kubernetes.io/projected/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-kube-api-access-ffjdk\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:24.959516 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959398 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7dg5\" (UniqueName: \"kubernetes.io/projected/b69c485c-88c0-43c5-a69c-d88c0f81fa22-kube-api-access-q7dg5\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:24.959516 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959429 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-dbus\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:24.959516 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959453 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-cnibin\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:24.959516 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959472 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-etc-openvswitch\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:24.959516 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959487 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-kubernetes\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:24.959516 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959502 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/80423b86-27d7-42d2-9daa-0399bf2303de-etc-tuned\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:24.960137 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959535 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9432e919-3bed-47a1-8b4e-4b979ebdb046-serviceca\") pod \"node-ca-h9qqf\" (UID: \"9432e919-3bed-47a1-8b4e-4b979ebdb046\") " pod="openshift-image-registry/node-ca-h9qqf" Oct 30 22:07:24.960137 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959562 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b69c485c-88c0-43c5-a69c-d88c0f81fa22-system-cni-dir\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:24.960137 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959585 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs\") pod \"network-metrics-daemon-l7k26\" (UID: \"f805f34e-7fc9-430b-9003-16d9765d6657\") " pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:24.960137 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959605 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9432e919-3bed-47a1-8b4e-4b979ebdb046-host\") pod \"node-ca-h9qqf\" (UID: \"9432e919-3bed-47a1-8b4e-4b979ebdb046\") " pod="openshift-image-registry/node-ca-h9qqf" Oct 30 22:07:24.960137 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959619 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-run-multus-certs\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:24.960137 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959645 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b69c485c-88c0-43c5-a69c-d88c0f81fa22-cnibin\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:24.960137 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959661 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b69c485c-88c0-43c5-a69c-d88c0f81fa22-cni-binary-copy\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:24.960137 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959697 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c135a307-61bc-4e34-9b5c-b97793255db7-hosts-file\") pod \"node-resolver-q5gw7\" (UID: \"c135a307-61bc-4e34-9b5c-b97793255db7\") " pod="openshift-dns/node-resolver-q5gw7" Oct 30 22:07:24.960137 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959724 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-run-ovn-kubernetes\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:24.960137 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959756 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:24.960137 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959782 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/95637308-9cdc-4a32-b573-70e1b4cff546-ovnkube-config\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:24.960137 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959806 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-cni-dir\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:24.960137 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959821 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-conf-dir\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:24.960137 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959839 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxhfx\" (UniqueName: \"kubernetes.io/projected/749dd349-e1d9-4b55-ab5c-5a30739a8cae-kube-api-access-gxhfx\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:24.960137 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959868 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-kubelet\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:24.960137 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959884 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-systemd-units\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:24.960794 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959909 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/95637308-9cdc-4a32-b573-70e1b4cff546-env-overrides\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:24.960794 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959964 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80423b86-27d7-42d2-9daa-0399bf2303de-tmp\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:24.960794 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.959993 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:24.960794 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.960036 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scwfv\" (UniqueName: \"kubernetes.io/projected/f805f34e-7fc9-430b-9003-16d9765d6657-kube-api-access-scwfv\") pod \"network-metrics-daemon-l7k26\" (UID: \"f805f34e-7fc9-430b-9003-16d9765d6657\") " pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:24.960794 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.960089 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-node-log\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:24.960794 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.960111 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-cni-bin\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:24.960794 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.960125 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-sysctl-conf\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:24.960794 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.960158 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-lib-modules\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:24.960794 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.960182 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8rz5\" (UniqueName: \"kubernetes.io/projected/9432e919-3bed-47a1-8b4e-4b979ebdb046-kube-api-access-m8rz5\") pod \"node-ca-h9qqf\" (UID: \"9432e919-3bed-47a1-8b4e-4b979ebdb046\") " pod="openshift-image-registry/node-ca-h9qqf" Oct 30 22:07:24.960794 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.960204 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-var-lib-cni-multus\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:24.960794 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.960231 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-var-lib-kubelet\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:24.960794 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.960268 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b69c485c-88c0-43c5-a69c-d88c0f81fa22-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:24.960794 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.960301 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-device-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:24.960794 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.960341 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-run\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:24.960794 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.960375 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-sys-fs\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:24.960794 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.960396 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-cni-netd\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:24.961327 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.960409 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-sys\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:24.961327 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.960423 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2mlc\" (UniqueName: \"kubernetes.io/projected/80423b86-27d7-42d2-9daa-0399bf2303de-kube-api-access-m2mlc\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:24.961327 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.960480 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-var-lib-cni-bin\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:24.961327 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.960521 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b69c485c-88c0-43c5-a69c-d88c0f81fa22-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:24.961327 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.960563 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-slash\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:24.961327 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.960603 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-run-systemd\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:24.961327 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.960641 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-kubelet-config\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:24.961327 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.960663 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/12b2045b-a302-47a1-b039-b8eadcc50c61-agent-certs\") pod \"konnectivity-agent-6qz5h\" (UID: \"12b2045b-a302-47a1-b039-b8eadcc50c61\") " pod="kube-system/konnectivity-agent-6qz5h" Oct 30 22:07:24.961327 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.960682 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-sysconfig\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:24.994409 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.994387 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 22:07:24.994540 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.994478 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 22:07:24.994540 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.994509 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 22:07:24.995019 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.994843 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 22:07:24.995019 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.994865 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qbtfs\"" Oct 30 22:07:24.995019 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.994887 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 22:07:24.995019 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.994892 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 22:07:24.995019 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.994940 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 22:07:24.995353 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.995065 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 22:07:24.995353 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.995068 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 22:07:24.995353 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.995130 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 22:07:24.995353 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.995139 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 22:07:24.995353 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.995195 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 22:07:24.995353 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.995253 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-vngtk\"" Oct 30 22:07:24.995353 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.995273 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zxnxl\"" Oct 30 22:07:24.995353 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.995301 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 22:07:24.995353 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.995321 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9wcbl\"" Oct 30 22:07:24.995353 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.995338 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 22:07:24.995353 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.995343 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nqvwl\"" Oct 30 22:07:24.995677 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.995381 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-xh2wl\"" Oct 30 22:07:24.995745 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:24.995722 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 22:07:25.061895 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.061809 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-var-lib-cni-bin\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.061895 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.061850 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b69c485c-88c0-43c5-a69c-d88c0f81fa22-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:25.061895 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.061875 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-slash\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.062151 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.061902 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-run-systemd\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.062151 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.061927 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-kubelet-config\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:25.062151 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.061947 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/12b2045b-a302-47a1-b039-b8eadcc50c61-agent-certs\") pod \"konnectivity-agent-6qz5h\" (UID: \"12b2045b-a302-47a1-b039-b8eadcc50c61\") " pod="kube-system/konnectivity-agent-6qz5h" Oct 30 22:07:25.062151 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.061939 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-var-lib-cni-bin\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.062151 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.061973 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-sysconfig\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.062151 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.061996 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-slash\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.062151 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062016 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-host\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.062151 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062017 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-run-systemd\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.062151 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062029 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-sysconfig\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.062151 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062038 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-kubelet-config\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:25.062151 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062069 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-run-netns\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.062151 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062101 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-host\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.062151 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062135 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c135a307-61bc-4e34-9b5c-b97793255db7-tmp-dir\") pod \"node-resolver-q5gw7\" (UID: \"c135a307-61bc-4e34-9b5c-b97793255db7\") " pod="openshift-dns/node-resolver-q5gw7" Oct 30 22:07:25.062151 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062155 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z8f9\" (UniqueName: \"kubernetes.io/projected/c135a307-61bc-4e34-9b5c-b97793255db7-kube-api-access-8z8f9\") pod \"node-resolver-q5gw7\" (UID: \"c135a307-61bc-4e34-9b5c-b97793255db7\") " pod="openshift-dns/node-resolver-q5gw7" Oct 30 22:07:25.062733 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062159 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-run-netns\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.062733 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062182 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-var-lib-kubelet\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.062733 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062203 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-run-ovn\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.062733 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062227 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-log-socket\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.062733 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062254 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-socket-dir-parent\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.062733 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062260 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-var-lib-kubelet\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.062733 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062291 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-system-cni-dir\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.062733 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062307 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-log-socket\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.062733 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062311 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-run-ovn\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.062733 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062343 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-socket-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:25.062733 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062366 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wrvl\" (UniqueName: \"kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl\") pod \"network-check-target-pmkb2\" (UID: \"d3d3d52b-f209-4ad4-b531-4fe19bad4bf1\") " pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:25.062733 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062386 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-var-lib-openvswitch\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.062733 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062407 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt5mm\" (UniqueName: \"kubernetes.io/projected/95637308-9cdc-4a32-b573-70e1b4cff546-kube-api-access-xt5mm\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.062733 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062423 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-system-cni-dir\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.062733 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062432 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-hostroot\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.062733 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062435 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-socket-dir-parent\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.062733 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062478 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-var-lib-openvswitch\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.062733 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062488 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-hostroot\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.063391 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062523 2579 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 22:07:25.063391 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062543 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c135a307-61bc-4e34-9b5c-b97793255db7-tmp-dir\") pod \"node-resolver-q5gw7\" (UID: \"c135a307-61bc-4e34-9b5c-b97793255db7\") " pod="openshift-dns/node-resolver-q5gw7" Oct 30 22:07:25.063391 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062615 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-registration-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:25.063391 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062646 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-run-netns\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.063391 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062667 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-run-openvswitch\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.063391 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062699 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-socket-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:25.063391 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062709 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-run-netns\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.063391 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062699 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-registration-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:25.063391 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062730 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-run-openvswitch\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.063391 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062741 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:25.063391 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062783 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b69c485c-88c0-43c5-a69c-d88c0f81fa22-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:25.063391 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062796 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/08904f35-ebda-4220-99c4-b2fb90aa2528-iptables-alerter-script\") pod \"iptables-alerter-zt9w6\" (UID: \"08904f35-ebda-4220-99c4-b2fb90aa2528\") " pod="openshift-network-operator/iptables-alerter-zt9w6" Oct 30 22:07:25.063391 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:25.062806 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:25.063391 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062828 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/95637308-9cdc-4a32-b573-70e1b4cff546-ovnkube-script-lib\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.063391 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062852 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-modprobe-d\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.063391 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:25.062878 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret podName:bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:25.56285051 +0000 UTC m=+2.055921298 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret") pod "global-pull-secret-syncer-gsvm9" (UID: "bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9") : object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:25.063391 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062908 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-sysctl-d\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.064010 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062931 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-systemd\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.064010 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062932 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-modprobe-d\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.064010 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062993 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-run-k8s-cni-cncf-io\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.064010 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.062997 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-systemd\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.064010 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063006 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-sysctl-d\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.064010 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063018 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/08904f35-ebda-4220-99c4-b2fb90aa2528-host-slash\") pod \"iptables-alerter-zt9w6\" (UID: \"08904f35-ebda-4220-99c4-b2fb90aa2528\") " pod="openshift-network-operator/iptables-alerter-zt9w6" Oct 30 22:07:25.064010 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063039 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-etc-kubernetes\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.064010 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063060 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-run-k8s-cni-cncf-io\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.064010 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063064 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/08904f35-ebda-4220-99c4-b2fb90aa2528-host-slash\") pod \"iptables-alerter-zt9w6\" (UID: \"08904f35-ebda-4220-99c4-b2fb90aa2528\") " pod="openshift-network-operator/iptables-alerter-zt9w6" Oct 30 22:07:25.064010 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063073 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-etc-selinux\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:25.064010 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063094 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2kvz\" (UniqueName: \"kubernetes.io/projected/08904f35-ebda-4220-99c4-b2fb90aa2528-kube-api-access-m2kvz\") pod \"iptables-alerter-zt9w6\" (UID: \"08904f35-ebda-4220-99c4-b2fb90aa2528\") " pod="openshift-network-operator/iptables-alerter-zt9w6" Oct 30 22:07:25.064010 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063101 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-etc-kubernetes\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.064010 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063115 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b69c485c-88c0-43c5-a69c-d88c0f81fa22-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:25.064010 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063138 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/95637308-9cdc-4a32-b573-70e1b4cff546-ovn-node-metrics-cert\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.064010 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063159 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/12b2045b-a302-47a1-b039-b8eadcc50c61-konnectivity-ca\") pod \"konnectivity-agent-6qz5h\" (UID: \"12b2045b-a302-47a1-b039-b8eadcc50c61\") " pod="kube-system/konnectivity-agent-6qz5h" Oct 30 22:07:25.064010 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063179 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-cni-binary-copy\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.064010 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063197 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-etc-selinux\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:25.064753 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063199 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-daemon-config\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.064753 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063294 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffjdk\" (UniqueName: \"kubernetes.io/projected/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-kube-api-access-ffjdk\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.064753 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063312 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7dg5\" (UniqueName: \"kubernetes.io/projected/b69c485c-88c0-43c5-a69c-d88c0f81fa22-kube-api-access-q7dg5\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:25.064753 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063328 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-dbus\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:25.064753 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063343 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-cnibin\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.064753 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063359 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-etc-openvswitch\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.064753 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063373 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-kubernetes\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.064753 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063389 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/80423b86-27d7-42d2-9daa-0399bf2303de-etc-tuned\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.064753 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063404 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9432e919-3bed-47a1-8b4e-4b979ebdb046-serviceca\") pod \"node-ca-h9qqf\" (UID: \"9432e919-3bed-47a1-8b4e-4b979ebdb046\") " pod="openshift-image-registry/node-ca-h9qqf" Oct 30 22:07:25.064753 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063425 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b69c485c-88c0-43c5-a69c-d88c0f81fa22-system-cni-dir\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:25.064753 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063448 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs\") pod \"network-metrics-daemon-l7k26\" (UID: \"f805f34e-7fc9-430b-9003-16d9765d6657\") " pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:25.064753 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063467 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9432e919-3bed-47a1-8b4e-4b979ebdb046-host\") pod \"node-ca-h9qqf\" (UID: \"9432e919-3bed-47a1-8b4e-4b979ebdb046\") " pod="openshift-image-registry/node-ca-h9qqf" Oct 30 22:07:25.064753 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063482 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-run-multus-certs\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.064753 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063501 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b69c485c-88c0-43c5-a69c-d88c0f81fa22-cnibin\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:25.064753 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063522 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b69c485c-88c0-43c5-a69c-d88c0f81fa22-cni-binary-copy\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:25.064753 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:25.063791 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:25.064753 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:25.063843 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs podName:f805f34e-7fc9-430b-9003-16d9765d6657 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:25.56382772 +0000 UTC m=+2.056898513 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs") pod "network-metrics-daemon-l7k26" (UID: "f805f34e-7fc9-430b-9003-16d9765d6657") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:25.065587 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063880 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c135a307-61bc-4e34-9b5c-b97793255db7-hosts-file\") pod \"node-resolver-q5gw7\" (UID: \"c135a307-61bc-4e34-9b5c-b97793255db7\") " pod="openshift-dns/node-resolver-q5gw7" Oct 30 22:07:25.065587 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063896 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b69c485c-88c0-43c5-a69c-d88c0f81fa22-system-cni-dir\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:25.065587 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063904 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-run-ovn-kubernetes\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.065587 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063932 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-run-ovn-kubernetes\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.065587 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063935 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.065587 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063985 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-etc-openvswitch\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.065587 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.063991 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-run-multus-certs\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.065587 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.064003 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-cnibin\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.065587 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.064033 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-kubernetes\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.065587 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.064058 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b69c485c-88c0-43c5-a69c-d88c0f81fa22-cnibin\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:25.065587 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.064066 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c135a307-61bc-4e34-9b5c-b97793255db7-hosts-file\") pod \"node-resolver-q5gw7\" (UID: \"c135a307-61bc-4e34-9b5c-b97793255db7\") " pod="openshift-dns/node-resolver-q5gw7" Oct 30 22:07:25.065587 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.064159 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-dbus\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:25.065587 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.064179 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/95637308-9cdc-4a32-b573-70e1b4cff546-ovnkube-config\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.065587 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.064207 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.065587 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.064766 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/95637308-9cdc-4a32-b573-70e1b4cff546-ovnkube-script-lib\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.065587 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.064206 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-cni-dir\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.065587 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.064896 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-cni-binary-copy\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.066311 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.064923 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/08904f35-ebda-4220-99c4-b2fb90aa2528-iptables-alerter-script\") pod \"iptables-alerter-zt9w6\" (UID: \"08904f35-ebda-4220-99c4-b2fb90aa2528\") " pod="openshift-network-operator/iptables-alerter-zt9w6" Oct 30 22:07:25.066311 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.064926 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-conf-dir\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.066311 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.064973 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-conf-dir\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.066311 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.064995 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxhfx\" (UniqueName: \"kubernetes.io/projected/749dd349-e1d9-4b55-ab5c-5a30739a8cae-kube-api-access-gxhfx\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:25.066311 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065020 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-kubelet\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.066311 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065062 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-systemd-units\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.066311 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065100 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/95637308-9cdc-4a32-b573-70e1b4cff546-env-overrides\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.066311 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065124 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80423b86-27d7-42d2-9daa-0399bf2303de-tmp\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.066311 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065147 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:25.066311 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065171 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scwfv\" (UniqueName: \"kubernetes.io/projected/f805f34e-7fc9-430b-9003-16d9765d6657-kube-api-access-scwfv\") pod \"network-metrics-daemon-l7k26\" (UID: \"f805f34e-7fc9-430b-9003-16d9765d6657\") " pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:25.066311 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065192 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-node-log\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.066311 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065214 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-cni-bin\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.066311 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065240 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-sysctl-conf\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.066311 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065260 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-lib-modules\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.066311 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065262 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/12b2045b-a302-47a1-b039-b8eadcc50c61-konnectivity-ca\") pod \"konnectivity-agent-6qz5h\" (UID: \"12b2045b-a302-47a1-b039-b8eadcc50c61\") " pod="kube-system/konnectivity-agent-6qz5h" Oct 30 22:07:25.066311 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065280 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8rz5\" (UniqueName: \"kubernetes.io/projected/9432e919-3bed-47a1-8b4e-4b979ebdb046-kube-api-access-m8rz5\") pod \"node-ca-h9qqf\" (UID: \"9432e919-3bed-47a1-8b4e-4b979ebdb046\") " pod="openshift-image-registry/node-ca-h9qqf" Oct 30 22:07:25.066311 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065303 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-var-lib-cni-multus\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.066884 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065341 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-var-lib-kubelet\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.066884 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065352 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b69c485c-88c0-43c5-a69c-d88c0f81fa22-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:25.066884 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065362 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b69c485c-88c0-43c5-a69c-d88c0f81fa22-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:25.066884 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065385 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-device-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:25.066884 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065387 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9432e919-3bed-47a1-8b4e-4b979ebdb046-serviceca\") pod \"node-ca-h9qqf\" (UID: \"9432e919-3bed-47a1-8b4e-4b979ebdb046\") " pod="openshift-image-registry/node-ca-h9qqf" Oct 30 22:07:25.066884 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065405 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-run\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.066884 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065430 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-sys-fs\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:25.066884 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065434 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9432e919-3bed-47a1-8b4e-4b979ebdb046-host\") pod \"node-ca-h9qqf\" (UID: \"9432e919-3bed-47a1-8b4e-4b979ebdb046\") " pod="openshift-image-registry/node-ca-h9qqf" Oct 30 22:07:25.066884 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065452 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-cni-netd\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.066884 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065476 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-sys\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.066884 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065491 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-cni-bin\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.066884 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065503 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2mlc\" (UniqueName: \"kubernetes.io/projected/80423b86-27d7-42d2-9daa-0399bf2303de-kube-api-access-m2mlc\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.066884 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065615 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-daemon-config\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.066884 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065649 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b69c485c-88c0-43c5-a69c-d88c0f81fa22-cni-binary-copy\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:25.066884 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065707 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-kubelet\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.066884 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065761 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:25.066884 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065760 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b69c485c-88c0-43c5-a69c-d88c0f81fa22-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:25.067464 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065793 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-device-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:25.067464 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065845 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-run\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.067464 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065873 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-sysctl-conf\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.067464 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065892 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-sys-fs\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:25.067464 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065964 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-lib-modules\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.067464 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065978 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-node-log\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.067464 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.066001 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-cni-netd\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.067464 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.066040 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-var-lib-cni-multus\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.067464 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.066066 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-sys\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.067464 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.065063 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-cni-dir\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.067464 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.066109 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-systemd-units\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.067464 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.066135 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-var-lib-kubelet\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.067464 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.066172 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/95637308-9cdc-4a32-b573-70e1b4cff546-ovnkube-config\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.067464 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.066426 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/95637308-9cdc-4a32-b573-70e1b4cff546-env-overrides\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.067464 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.066929 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/80423b86-27d7-42d2-9daa-0399bf2303de-etc-tuned\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.067464 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.066964 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/12b2045b-a302-47a1-b039-b8eadcc50c61-agent-certs\") pod \"konnectivity-agent-6qz5h\" (UID: \"12b2045b-a302-47a1-b039-b8eadcc50c61\") " pod="kube-system/konnectivity-agent-6qz5h" Oct 30 22:07:25.067464 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.066980 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/95637308-9cdc-4a32-b573-70e1b4cff546-ovn-node-metrics-cert\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.068132 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.068110 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80423b86-27d7-42d2-9daa-0399bf2303de-tmp\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.079879 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:25.079855 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 22:07:25.079879 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:25.079881 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 22:07:25.080057 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:25.079894 2579 projected.go:196] Error preparing data for projected volume kube-api-access-4wrvl for pod openshift-network-diagnostics/network-check-target-pmkb2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:25.080057 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:25.079980 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl podName:d3d3d52b-f209-4ad4-b531-4fe19bad4bf1 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:25.579961028 +0000 UTC m=+2.073031816 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4wrvl" (UniqueName: "kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl") pod "network-check-target-pmkb2" (UID: "d3d3d52b-f209-4ad4-b531-4fe19bad4bf1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:25.082603 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.082578 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z8f9\" (UniqueName: \"kubernetes.io/projected/c135a307-61bc-4e34-9b5c-b97793255db7-kube-api-access-8z8f9\") pod \"node-resolver-q5gw7\" (UID: \"c135a307-61bc-4e34-9b5c-b97793255db7\") " pod="openshift-dns/node-resolver-q5gw7" Oct 30 22:07:25.082753 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.082728 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2kvz\" (UniqueName: \"kubernetes.io/projected/08904f35-ebda-4220-99c4-b2fb90aa2528-kube-api-access-m2kvz\") pod \"iptables-alerter-zt9w6\" (UID: \"08904f35-ebda-4220-99c4-b2fb90aa2528\") " pod="openshift-network-operator/iptables-alerter-zt9w6" Oct 30 22:07:25.083926 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.083892 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffjdk\" (UniqueName: \"kubernetes.io/projected/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-kube-api-access-ffjdk\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.084146 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.084120 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7dg5\" (UniqueName: \"kubernetes.io/projected/b69c485c-88c0-43c5-a69c-d88c0f81fa22-kube-api-access-q7dg5\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:25.084538 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.084514 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxhfx\" (UniqueName: \"kubernetes.io/projected/749dd349-e1d9-4b55-ab5c-5a30739a8cae-kube-api-access-gxhfx\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:25.085015 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.084993 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt5mm\" (UniqueName: \"kubernetes.io/projected/95637308-9cdc-4a32-b573-70e1b4cff546-kube-api-access-xt5mm\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.085122 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.085080 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scwfv\" (UniqueName: \"kubernetes.io/projected/f805f34e-7fc9-430b-9003-16d9765d6657-kube-api-access-scwfv\") pod \"network-metrics-daemon-l7k26\" (UID: \"f805f34e-7fc9-430b-9003-16d9765d6657\") " pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:25.085301 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.085280 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2mlc\" (UniqueName: \"kubernetes.io/projected/80423b86-27d7-42d2-9daa-0399bf2303de-kube-api-access-m2mlc\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.085512 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.085497 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8rz5\" (UniqueName: \"kubernetes.io/projected/9432e919-3bed-47a1-8b4e-4b979ebdb046-kube-api-access-m8rz5\") pod \"node-ca-h9qqf\" (UID: \"9432e919-3bed-47a1-8b4e-4b979ebdb046\") " pod="openshift-image-registry/node-ca-h9qqf" Oct 30 22:07:25.119511 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:25.119481 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00a20884b67d7fdeb7bbc7e471370a8a.slice/crio-475de44354d8527225668c6098e5464b1da922d27afa961e9923b4865768fb6f WatchSource:0}: Error finding container 475de44354d8527225668c6098e5464b1da922d27afa961e9923b4865768fb6f: Status 404 returned error can't find the container with id 475de44354d8527225668c6098e5464b1da922d27afa961e9923b4865768fb6f Oct 30 22:07:25.119921 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:25.119899 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf39ce36b79a5ed24b44c91f0e6e003f7.slice/crio-b06c9fab80ab4df18d7da9f617e738d96d9361cf364e83dd6fd16fa5a92923b2 WatchSource:0}: Error finding container b06c9fab80ab4df18d7da9f617e738d96d9361cf364e83dd6fd16fa5a92923b2: Status 404 returned error can't find the container with id b06c9fab80ab4df18d7da9f617e738d96d9361cf364e83dd6fd16fa5a92923b2 Oct 30 22:07:25.124267 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.124252 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 22:07:25.183177 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.183146 2579 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 22:07:25.214610 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.214580 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h9qqf" Oct 30 22:07:25.224970 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:25.224937 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9432e919_3bed_47a1_8b4e_4b979ebdb046.slice/crio-4d7c4a4a9633288554efa3923877c1f2118d6d74c526f92b7752344437deefb5 WatchSource:0}: Error finding container 4d7c4a4a9633288554efa3923877c1f2118d6d74c526f92b7752344437deefb5: Status 404 returned error can't find the container with id 4d7c4a4a9633288554efa3923877c1f2118d6d74c526f92b7752344437deefb5 Oct 30 22:07:25.235149 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.235124 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zt9w6" Oct 30 22:07:25.243440 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:25.243414 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08904f35_ebda_4220_99c4_b2fb90aa2528.slice/crio-18f67f9fdfc9f0184b71fce97c310c9a17144518445298f23b5e84a23de2dc6e WatchSource:0}: Error finding container 18f67f9fdfc9f0184b71fce97c310c9a17144518445298f23b5e84a23de2dc6e: Status 404 returned error can't find the container with id 18f67f9fdfc9f0184b71fce97c310c9a17144518445298f23b5e84a23de2dc6e Oct 30 22:07:25.272272 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.272239 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:25.273293 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.273277 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-6qz5h" Oct 30 22:07:25.278166 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.278146 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4gdsr" Oct 30 22:07:25.283008 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:25.282988 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95637308_9cdc_4a32_b573_70e1b4cff546.slice/crio-dfe98ac968ba7cfb8796c8d56d5f9dc5e8daf9f32802ddaa6ea4d95ebad9cf72 WatchSource:0}: Error finding container dfe98ac968ba7cfb8796c8d56d5f9dc5e8daf9f32802ddaa6ea4d95ebad9cf72: Status 404 returned error can't find the container with id dfe98ac968ba7cfb8796c8d56d5f9dc5e8daf9f32802ddaa6ea4d95ebad9cf72 Oct 30 22:07:25.284337 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:25.284311 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12b2045b_a302_47a1_b039_b8eadcc50c61.slice/crio-fe78c30b59feca53bbcfe1307dc8221a5ca0fd965907fc588da59f0e591181ae WatchSource:0}: Error finding container fe78c30b59feca53bbcfe1307dc8221a5ca0fd965907fc588da59f0e591181ae: Status 404 returned error can't find the container with id fe78c30b59feca53bbcfe1307dc8221a5ca0fd965907fc588da59f0e591181ae Oct 30 22:07:25.289285 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:25.289263 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9de98c42_a20f_4d6f_b8ab_d0cabf70dedf.slice/crio-b394b3d88a7bd12547390ee081799d630407f6779fee7d5c52d926a8fcb48fca WatchSource:0}: Error finding container b394b3d88a7bd12547390ee081799d630407f6779fee7d5c52d926a8fcb48fca: Status 404 returned error can't find the container with id b394b3d88a7bd12547390ee081799d630407f6779fee7d5c52d926a8fcb48fca Oct 30 22:07:25.303546 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.303519 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:07:25.309671 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.309652 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:07:25.313156 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:25.313137 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb69c485c_88c0_43c5_a69c_d88c0f81fa22.slice/crio-ab2ecd2f1b5cc900dd679aa3d840dd78546c4e62e7833438c44ce0b6a7ce7731 WatchSource:0}: Error finding container ab2ecd2f1b5cc900dd679aa3d840dd78546c4e62e7833438c44ce0b6a7ce7731: Status 404 returned error can't find the container with id ab2ecd2f1b5cc900dd679aa3d840dd78546c4e62e7833438c44ce0b6a7ce7731 Oct 30 22:07:25.318229 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:25.318207 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod749dd349_e1d9_4b55_ab5c_5a30739a8cae.slice/crio-0172a49ee3a66d8484ba34ef41ebc50fa138bddb747ecdca0564ed33797251fd WatchSource:0}: Error finding container 0172a49ee3a66d8484ba34ef41ebc50fa138bddb747ecdca0564ed33797251fd: Status 404 returned error can't find the container with id 0172a49ee3a66d8484ba34ef41ebc50fa138bddb747ecdca0564ed33797251fd Oct 30 22:07:25.320300 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.320216 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:07:25.326359 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.326336 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q5gw7" Oct 30 22:07:25.335844 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:25.335820 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc135a307_61bc_4e34_9b5c_b97793255db7.slice/crio-f2a1d08519517fcb1d6fe59040ac4cba8e8b764e6ccee10f12cb664f4130f603 WatchSource:0}: Error finding container f2a1d08519517fcb1d6fe59040ac4cba8e8b764e6ccee10f12cb664f4130f603: Status 404 returned error can't find the container with id f2a1d08519517fcb1d6fe59040ac4cba8e8b764e6ccee10f12cb664f4130f603 Oct 30 22:07:25.376006 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.375968 2579 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 22:07:25.569560 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.569464 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:25.569560 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.569512 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs\") pod \"network-metrics-daemon-l7k26\" (UID: \"f805f34e-7fc9-430b-9003-16d9765d6657\") " pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:25.569784 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:25.569650 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:25.569784 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:25.569675 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:25.569784 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:25.569717 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs podName:f805f34e-7fc9-430b-9003-16d9765d6657 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:26.569699069 +0000 UTC m=+3.062769851 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs") pod "network-metrics-daemon-l7k26" (UID: "f805f34e-7fc9-430b-9003-16d9765d6657") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:25.569784 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:25.569738 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret podName:bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:26.569727716 +0000 UTC m=+3.062798501 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret") pod "global-pull-secret-syncer-gsvm9" (UID: "bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9") : object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:25.670980 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.670319 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wrvl\" (UniqueName: \"kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl\") pod \"network-check-target-pmkb2\" (UID: \"d3d3d52b-f209-4ad4-b531-4fe19bad4bf1\") " pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:25.670980 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:25.670537 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 22:07:25.670980 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:25.670558 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 22:07:25.670980 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:25.670570 2579 projected.go:196] Error preparing data for projected volume kube-api-access-4wrvl for pod openshift-network-diagnostics/network-check-target-pmkb2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:25.670980 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:25.670636 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl podName:d3d3d52b-f209-4ad4-b531-4fe19bad4bf1 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:26.670612644 +0000 UTC m=+3.163683419 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wrvl" (UniqueName: "kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl") pod "network-check-target-pmkb2" (UID: "d3d3d52b-f209-4ad4-b531-4fe19bad4bf1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:25.907976 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.907863 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 22:02:24 +0000 UTC" deadline="2028-04-14 14:28:27.052577253 +0000 UTC" Oct 30 22:07:25.907976 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:25.907927 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12760h21m1.144653979s" Oct 30 22:07:26.028590 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:26.028525 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q5gw7" event={"ID":"c135a307-61bc-4e34-9b5c-b97793255db7","Type":"ContainerStarted","Data":"f2a1d08519517fcb1d6fe59040ac4cba8e8b764e6ccee10f12cb664f4130f603"} Oct 30 22:07:26.047785 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:26.047655 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w692c" event={"ID":"80423b86-27d7-42d2-9daa-0399bf2303de","Type":"ContainerStarted","Data":"e1dc7d4424fe079dbc77bfd4ae6252885246ed1fbfc0f0401a843aa231007292"} Oct 30 22:07:26.066445 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:26.066406 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4gdsr" event={"ID":"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf","Type":"ContainerStarted","Data":"b394b3d88a7bd12547390ee081799d630407f6779fee7d5c52d926a8fcb48fca"} Oct 30 22:07:26.079268 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:26.079229 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6qz5h" event={"ID":"12b2045b-a302-47a1-b039-b8eadcc50c61","Type":"ContainerStarted","Data":"fe78c30b59feca53bbcfe1307dc8221a5ca0fd965907fc588da59f0e591181ae"} Oct 30 22:07:26.084816 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:26.084779 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zt9w6" event={"ID":"08904f35-ebda-4220-99c4-b2fb90aa2528","Type":"ContainerStarted","Data":"18f67f9fdfc9f0184b71fce97c310c9a17144518445298f23b5e84a23de2dc6e"} Oct 30 22:07:26.100962 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:26.100924 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" event={"ID":"749dd349-e1d9-4b55-ab5c-5a30739a8cae","Type":"ContainerStarted","Data":"0172a49ee3a66d8484ba34ef41ebc50fa138bddb747ecdca0564ed33797251fd"} Oct 30 22:07:26.119755 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:26.119703 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lzh8c" event={"ID":"b69c485c-88c0-43c5-a69c-d88c0f81fa22","Type":"ContainerStarted","Data":"ab2ecd2f1b5cc900dd679aa3d840dd78546c4e62e7833438c44ce0b6a7ce7731"} Oct 30 22:07:26.122933 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:26.122897 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" event={"ID":"95637308-9cdc-4a32-b573-70e1b4cff546","Type":"ContainerStarted","Data":"dfe98ac968ba7cfb8796c8d56d5f9dc5e8daf9f32802ddaa6ea4d95ebad9cf72"} Oct 30 22:07:26.141706 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:26.141664 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h9qqf" event={"ID":"9432e919-3bed-47a1-8b4e-4b979ebdb046","Type":"ContainerStarted","Data":"4d7c4a4a9633288554efa3923877c1f2118d6d74c526f92b7752344437deefb5"} Oct 30 22:07:26.160316 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:26.160219 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" event={"ID":"f39ce36b79a5ed24b44c91f0e6e003f7","Type":"ContainerStarted","Data":"b06c9fab80ab4df18d7da9f617e738d96d9361cf364e83dd6fd16fa5a92923b2"} Oct 30 22:07:26.183308 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:26.183268 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" event={"ID":"00a20884b67d7fdeb7bbc7e471370a8a","Type":"ContainerStarted","Data":"475de44354d8527225668c6098e5464b1da922d27afa961e9923b4865768fb6f"} Oct 30 22:07:26.214116 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:26.214063 2579 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 22:07:26.578856 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:26.578767 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:26.578856 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:26.578827 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs\") pod \"network-metrics-daemon-l7k26\" (UID: \"f805f34e-7fc9-430b-9003-16d9765d6657\") " pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:26.579120 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:26.578989 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:26.579120 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:26.579072 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs podName:f805f34e-7fc9-430b-9003-16d9765d6657 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:28.579035615 +0000 UTC m=+5.072106392 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs") pod "network-metrics-daemon-l7k26" (UID: "f805f34e-7fc9-430b-9003-16d9765d6657") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:26.579529 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:26.579511 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:26.579588 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:26.579569 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret podName:bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:28.579552977 +0000 UTC m=+5.072623768 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret") pod "global-pull-secret-syncer-gsvm9" (UID: "bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9") : object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:26.682513 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:26.682463 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wrvl\" (UniqueName: \"kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl\") pod \"network-check-target-pmkb2\" (UID: \"d3d3d52b-f209-4ad4-b531-4fe19bad4bf1\") " pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:26.682709 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:26.682679 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 22:07:26.682709 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:26.682703 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 22:07:26.682815 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:26.682715 2579 projected.go:196] Error preparing data for projected volume kube-api-access-4wrvl for pod openshift-network-diagnostics/network-check-target-pmkb2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:26.682815 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:26.682779 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl podName:d3d3d52b-f209-4ad4-b531-4fe19bad4bf1 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:28.682761286 +0000 UTC m=+5.175832067 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wrvl" (UniqueName: "kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl") pod "network-check-target-pmkb2" (UID: "d3d3d52b-f209-4ad4-b531-4fe19bad4bf1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:26.908886 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:26.908765 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 22:02:24 +0000 UTC" deadline="2028-07-11 18:29:16.67069598 +0000 UTC" Oct 30 22:07:26.908886 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:26.908833 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14876h21m49.761867739s" Oct 30 22:07:26.996136 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:26.996095 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:26.996330 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:26.996231 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsvm9" podUID="bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9" Oct 30 22:07:26.996330 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:26.996324 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:26.996454 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:26.996410 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7k26" podUID="f805f34e-7fc9-430b-9003-16d9765d6657" Oct 30 22:07:26.996507 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:26.996483 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:26.996560 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:26.996545 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmkb2" podUID="d3d3d52b-f209-4ad4-b531-4fe19bad4bf1" Oct 30 22:07:28.603448 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:28.603390 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs\") pod \"network-metrics-daemon-l7k26\" (UID: \"f805f34e-7fc9-430b-9003-16d9765d6657\") " pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:28.603931 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:28.603484 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:28.603931 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:28.603623 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:28.603931 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:28.603697 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret podName:bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:32.603669238 +0000 UTC m=+9.096740032 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret") pod "global-pull-secret-syncer-gsvm9" (UID: "bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9") : object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:28.604537 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:28.604121 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:28.604537 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:28.604176 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs podName:f805f34e-7fc9-430b-9003-16d9765d6657 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:32.604161015 +0000 UTC m=+9.097231806 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs") pod "network-metrics-daemon-l7k26" (UID: "f805f34e-7fc9-430b-9003-16d9765d6657") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:28.704542 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:28.704500 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wrvl\" (UniqueName: \"kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl\") pod \"network-check-target-pmkb2\" (UID: \"d3d3d52b-f209-4ad4-b531-4fe19bad4bf1\") " pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:28.704792 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:28.704706 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 22:07:28.704792 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:28.704730 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 22:07:28.704792 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:28.704743 2579 projected.go:196] Error preparing data for projected volume kube-api-access-4wrvl for pod openshift-network-diagnostics/network-check-target-pmkb2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:28.704970 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:28.704802 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl podName:d3d3d52b-f209-4ad4-b531-4fe19bad4bf1 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:32.704783301 +0000 UTC m=+9.197854097 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wrvl" (UniqueName: "kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl") pod "network-check-target-pmkb2" (UID: "d3d3d52b-f209-4ad4-b531-4fe19bad4bf1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:28.997194 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:28.996355 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:28.997194 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:28.996496 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7k26" podUID="f805f34e-7fc9-430b-9003-16d9765d6657" Oct 30 22:07:28.997194 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:28.996913 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:28.997194 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:28.996998 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmkb2" podUID="d3d3d52b-f209-4ad4-b531-4fe19bad4bf1" Oct 30 22:07:28.997194 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:28.997059 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:28.997194 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:28.997159 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsvm9" podUID="bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9" Oct 30 22:07:29.213729 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:29.212435 2579 generic.go:356] "Generic (PLEG): container finished" podID="b69c485c-88c0-43c5-a69c-d88c0f81fa22" containerID="a303caee0480d05cccf6109f690fa1715f1337d7954b61e643478f70dbd51148" exitCode=0 Oct 30 22:07:29.213729 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:29.212492 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lzh8c" event={"ID":"b69c485c-88c0-43c5-a69c-d88c0f81fa22","Type":"ContainerDied","Data":"a303caee0480d05cccf6109f690fa1715f1337d7954b61e643478f70dbd51148"} Oct 30 22:07:30.218877 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:30.217847 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" event={"ID":"f39ce36b79a5ed24b44c91f0e6e003f7","Type":"ContainerStarted","Data":"3bb33b8d00383e9f2982a9dde4b652ba8dd5ec9ffe8cd9ea54e5c41f0f694f1f"} Oct 30 22:07:30.224305 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:30.223565 2579 generic.go:356] "Generic (PLEG): container finished" podID="00a20884b67d7fdeb7bbc7e471370a8a" containerID="297f568d42424df7a502b35ac0d6b8dd3dadc151f32d4205f6a783c16e5d4013" exitCode=0 Oct 30 22:07:30.224305 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:30.223653 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" event={"ID":"00a20884b67d7fdeb7bbc7e471370a8a","Type":"ContainerDied","Data":"297f568d42424df7a502b35ac0d6b8dd3dadc151f32d4205f6a783c16e5d4013"} Oct 30 22:07:30.235152 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:30.235121 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-6qz5h" event={"ID":"12b2045b-a302-47a1-b039-b8eadcc50c61","Type":"ContainerStarted","Data":"21f128fc09f8d9dbb0ded5fd8da04adee4e2a4a4b16f2b8574df74a94a6bdb5d"} Oct 30 22:07:30.291697 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:30.291621 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" podStartSLOduration=6.291597509 podStartE2EDuration="6.291597509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:30.254814984 +0000 UTC m=+6.747885781" watchObservedRunningTime="2026-10-30 22:07:30.291597509 +0000 UTC m=+6.784668308" Oct 30 22:07:30.334824 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:30.334764 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-6qz5h" podStartSLOduration=2.046649774 podStartE2EDuration="6.334745863s" totalImagesPullingTime="4.288096089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:30.334296606 +0000 UTC m=+6.827367406" watchObservedRunningTime="2026-10-30 22:07:30.334745863 +0000 UTC m=+6.827816663" Oct 30 22:07:30.995849 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:30.995818 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:30.996022 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:30.995957 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7k26" podUID="f805f34e-7fc9-430b-9003-16d9765d6657" Oct 30 22:07:30.996217 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:30.996195 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:30.996319 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:30.996296 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsvm9" podUID="bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9" Oct 30 22:07:30.996319 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:30.996308 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:30.996427 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:30.996379 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmkb2" podUID="d3d3d52b-f209-4ad4-b531-4fe19bad4bf1" Oct 30 22:07:31.154976 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:31.154882 2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-6qz5h" Oct 30 22:07:31.155487 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:31.155464 2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-6qz5h" Oct 30 22:07:31.257240 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:31.257192 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h9qqf" event={"ID":"9432e919-3bed-47a1-8b4e-4b979ebdb046","Type":"ContainerStarted","Data":"0a40bbe9e5167141cf5b3aa6c411e302e1622e3db2848bf494fba4eec6d2d6b7"} Oct 30 22:07:31.261182 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:31.261156 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" event={"ID":"00a20884b67d7fdeb7bbc7e471370a8a","Type":"ContainerStarted","Data":"fa5bf0103648429155c90d187a318e250d1f61b09d5bbc330afeea5e20e37984"} Oct 30 22:07:31.261938 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:31.261645 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-6qz5h" Oct 30 22:07:31.262067 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:31.262022 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6qz5h" Oct 30 22:07:31.334208 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:31.334156 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-h9qqf" podStartSLOduration=1.711072479 podStartE2EDuration="7.334135626s" totalImagesPullingTime="5.623063147s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:31.29603426 +0000 UTC m=+7.789105059" watchObservedRunningTime="2026-10-30 22:07:31.334135626 +0000 UTC m=+7.827206427" Oct 30 22:07:31.375329 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:31.375267 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" podStartSLOduration=7.375245575 podStartE2EDuration="7.375245575s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:31.334968285 +0000 UTC m=+7.828039109" watchObservedRunningTime="2026-10-30 22:07:31.375245575 +0000 UTC m=+7.868316372" Oct 30 22:07:32.270037 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:32.269998 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" event={"ID":"749dd349-e1d9-4b55-ab5c-5a30739a8cae","Type":"ContainerStarted","Data":"b36c76e9fc407b4f31979c2d4d92eebb11ccecd4adfa73dc026c41d7706999d1"} Oct 30 22:07:32.645646 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:32.645558 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:32.645646 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:32.645624 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs\") pod \"network-metrics-daemon-l7k26\" (UID: \"f805f34e-7fc9-430b-9003-16d9765d6657\") " pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:32.645878 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:32.645801 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:32.645929 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:32.645919 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs podName:f805f34e-7fc9-430b-9003-16d9765d6657 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:40.64590064 +0000 UTC m=+17.138971429 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs") pod "network-metrics-daemon-l7k26" (UID: "f805f34e-7fc9-430b-9003-16d9765d6657") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:32.646503 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:32.646365 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:32.646503 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:32.646421 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret podName:bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:40.646405731 +0000 UTC m=+17.139476507 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret") pod "global-pull-secret-syncer-gsvm9" (UID: "bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9") : object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:32.746757 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:32.746705 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wrvl\" (UniqueName: \"kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl\") pod \"network-check-target-pmkb2\" (UID: \"d3d3d52b-f209-4ad4-b531-4fe19bad4bf1\") " pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:32.746919 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:32.746905 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 22:07:32.746965 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:32.746923 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 22:07:32.746965 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:32.746937 2579 projected.go:196] Error preparing data for projected volume kube-api-access-4wrvl for pod openshift-network-diagnostics/network-check-target-pmkb2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:32.747031 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:32.747000 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl podName:d3d3d52b-f209-4ad4-b531-4fe19bad4bf1 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:40.746975223 +0000 UTC m=+17.240046004 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wrvl" (UniqueName: "kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl") pod "network-check-target-pmkb2" (UID: "d3d3d52b-f209-4ad4-b531-4fe19bad4bf1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:32.995821 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:32.995755 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:32.995994 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:32.995855 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmkb2" podUID="d3d3d52b-f209-4ad4-b531-4fe19bad4bf1" Oct 30 22:07:32.995994 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:32.995930 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:32.996143 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:32.995999 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsvm9" podUID="bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9" Oct 30 22:07:32.996143 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:32.996081 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:32.996215 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:32.996169 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7k26" podUID="f805f34e-7fc9-430b-9003-16d9765d6657" Oct 30 22:07:33.230731 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:33.230701 2579 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 22:07:33.278179 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:33.278094 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" event={"ID":"749dd349-e1d9-4b55-ab5c-5a30739a8cae","Type":"ContainerStarted","Data":"3644994b6ec995eada829ecfd9ba91e82243783f782c72fe6af62c09c5b18332"} Oct 30 22:07:33.921487 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:33.920911 2579 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T22:07:33.230716989Z","UUID":"65df7519-7423-44e6-96d1-944763a4d02b","Handler":null,"Name":"","Endpoint":""} Oct 30 22:07:33.922666 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:33.922648 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 22:07:33.922749 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:33.922676 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 22:07:34.282053 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:34.282013 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q5gw7" event={"ID":"c135a307-61bc-4e34-9b5c-b97793255db7","Type":"ContainerStarted","Data":"68fd6d6a8869419f95ad980ad852ce0a86c8fcbbe7958b0aa73ca46577fe3e87"} Oct 30 22:07:34.284062 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:34.284011 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4gdsr" event={"ID":"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf","Type":"ContainerStarted","Data":"5daf6a7e1a757a6b298534f7992d3f03de63f179e51b801807f14c1298f878fe"} Oct 30 22:07:34.286773 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:34.286740 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lzh8c" event={"ID":"b69c485c-88c0-43c5-a69c-d88c0f81fa22","Type":"ContainerStarted","Data":"53542bbdbecb2d73c6ed64bc560df3290657396e85bdf49443db1c30453f27de"} Oct 30 22:07:34.319518 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:34.319461 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-q5gw7" podStartSLOduration=1.9461751010000001 podStartE2EDuration="10.31944098s" totalImagesPullingTime="8.373265879s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:34.31889394 +0000 UTC m=+10.811964737" watchObservedRunningTime="2026-10-30 22:07:34.31944098 +0000 UTC m=+10.812511777" Oct 30 22:07:34.399271 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:34.399217 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-4gdsr" podStartSLOduration=1.966797799 podStartE2EDuration="10.399198092s" totalImagesPullingTime="8.432400293s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:34.358152597 +0000 UTC m=+10.851223393" watchObservedRunningTime="2026-10-30 22:07:34.399198092 +0000 UTC m=+10.892268889" Oct 30 22:07:34.996360 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:34.996116 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:34.996539 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:34.996467 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsvm9" podUID="bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9" Oct 30 22:07:34.996539 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:34.996184 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:34.996680 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:34.996557 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmkb2" podUID="d3d3d52b-f209-4ad4-b531-4fe19bad4bf1" Oct 30 22:07:34.996680 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:34.996148 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:34.996680 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:34.996621 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7k26" podUID="f805f34e-7fc9-430b-9003-16d9765d6657" Oct 30 22:07:35.292520 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:35.292442 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" event={"ID":"749dd349-e1d9-4b55-ab5c-5a30739a8cae","Type":"ContainerStarted","Data":"d2138b2d2504ebec08f2ba9965b7bd585619db900777c688e5c2fe4210f1d923"} Oct 30 22:07:35.295457 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:35.295425 2579 generic.go:356] "Generic (PLEG): container finished" podID="b69c485c-88c0-43c5-a69c-d88c0f81fa22" containerID="53542bbdbecb2d73c6ed64bc560df3290657396e85bdf49443db1c30453f27de" exitCode=0 Oct 30 22:07:35.295781 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:35.295758 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lzh8c" event={"ID":"b69c485c-88c0-43c5-a69c-d88c0f81fa22","Type":"ContainerDied","Data":"53542bbdbecb2d73c6ed64bc560df3290657396e85bdf49443db1c30453f27de"} Oct 30 22:07:35.298165 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:35.298145 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zt9w6" event={"ID":"08904f35-ebda-4220-99c4-b2fb90aa2528","Type":"ContainerStarted","Data":"aafb7bae570d6847c259e721daa189d735e144af0c088c303c2931ace5824e44"} Oct 30 22:07:35.335008 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:35.334944 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" podStartSLOduration=2.718550467 podStartE2EDuration="11.334924719s" totalImagesPullingTime="8.616374252s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:35.334032075 +0000 UTC m=+11.827102873" watchObservedRunningTime="2026-10-30 22:07:35.334924719 +0000 UTC m=+11.827995517" Oct 30 22:07:35.371570 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:35.371510 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zt9w6" podStartSLOduration=3.072515988 podStartE2EDuration="11.371490701s" totalImagesPullingTime="8.298974713s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:35.370450279 +0000 UTC m=+11.863521076" watchObservedRunningTime="2026-10-30 22:07:35.371490701 +0000 UTC m=+11.864561498" Oct 30 22:07:36.302757 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:36.302726 2579 generic.go:356] "Generic (PLEG): container finished" podID="b69c485c-88c0-43c5-a69c-d88c0f81fa22" containerID="177c2909abaca0af49fd5adeec9fc16e609b38bd8a95136dca8772dbb8172abb" exitCode=0 Oct 30 22:07:36.303378 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:36.302813 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lzh8c" event={"ID":"b69c485c-88c0-43c5-a69c-d88c0f81fa22","Type":"ContainerDied","Data":"177c2909abaca0af49fd5adeec9fc16e609b38bd8a95136dca8772dbb8172abb"} Oct 30 22:07:36.304453 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:36.304424 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w692c" event={"ID":"80423b86-27d7-42d2-9daa-0399bf2303de","Type":"ContainerStarted","Data":"2f01e86b955b7030b8e72ab2395a25c4156d8969da82e36b7bce9508a9d702c4"} Oct 30 22:07:36.383817 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:36.383722 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-w692c" podStartSLOduration=1.924455848 podStartE2EDuration="12.383705372s" totalImagesPullingTime="10.459249524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:36.383456983 +0000 UTC m=+12.876527781" watchObservedRunningTime="2026-10-30 22:07:36.383705372 +0000 UTC m=+12.876776168" Oct 30 22:07:36.995923 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:36.995886 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:36.996123 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:36.995974 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:36.996123 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:36.995998 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:36.996221 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:36.996131 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsvm9" podUID="bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9" Oct 30 22:07:36.996284 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:36.996263 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7k26" podUID="f805f34e-7fc9-430b-9003-16d9765d6657" Oct 30 22:07:36.996362 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:36.996347 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmkb2" podUID="d3d3d52b-f209-4ad4-b531-4fe19bad4bf1" Oct 30 22:07:38.315094 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:38.315058 2579 generic.go:356] "Generic (PLEG): container finished" podID="b69c485c-88c0-43c5-a69c-d88c0f81fa22" containerID="a9d3d2a7cb4c13878d20d79b20c82781a4d7fea6517cc581c24f8ad94c0457b3" exitCode=0 Oct 30 22:07:38.315628 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:38.315103 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lzh8c" event={"ID":"b69c485c-88c0-43c5-a69c-d88c0f81fa22","Type":"ContainerDied","Data":"a9d3d2a7cb4c13878d20d79b20c82781a4d7fea6517cc581c24f8ad94c0457b3"} Oct 30 22:07:38.995810 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:38.995570 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:38.996004 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:38.995616 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:38.996004 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:38.995909 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmkb2" podUID="d3d3d52b-f209-4ad4-b531-4fe19bad4bf1" Oct 30 22:07:38.996004 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:38.995616 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:38.996171 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:38.996009 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7k26" podUID="f805f34e-7fc9-430b-9003-16d9765d6657" Oct 30 22:07:38.996171 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:38.996071 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsvm9" podUID="bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9" Oct 30 22:07:40.706086 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:40.705896 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:40.706372 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:40.706109 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs\") pod \"network-metrics-daemon-l7k26\" (UID: \"f805f34e-7fc9-430b-9003-16d9765d6657\") " pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:40.706372 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:40.706065 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:40.706372 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:40.706206 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:40.706372 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:40.706233 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret podName:bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:56.706202498 +0000 UTC m=+33.199273285 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret") pod "global-pull-secret-syncer-gsvm9" (UID: "bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9") : object "kube-system"/"original-pull-secret" not registered Oct 30 22:07:40.706372 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:40.706253 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs podName:f805f34e-7fc9-430b-9003-16d9765d6657 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:56.706243045 +0000 UTC m=+33.199313826 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs") pod "network-metrics-daemon-l7k26" (UID: "f805f34e-7fc9-430b-9003-16d9765d6657") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 22:07:40.806811 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:40.806788 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wrvl\" (UniqueName: \"kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl\") pod \"network-check-target-pmkb2\" (UID: \"d3d3d52b-f209-4ad4-b531-4fe19bad4bf1\") " pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:40.806982 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:40.806964 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 22:07:40.807037 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:40.806989 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 22:07:40.807037 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:40.806999 2579 projected.go:196] Error preparing data for projected volume kube-api-access-4wrvl for pod openshift-network-diagnostics/network-check-target-pmkb2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:40.807167 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:40.807070 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl podName:d3d3d52b-f209-4ad4-b531-4fe19bad4bf1 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:56.80703553 +0000 UTC m=+33.300106317 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4wrvl" (UniqueName: "kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl") pod "network-check-target-pmkb2" (UID: "d3d3d52b-f209-4ad4-b531-4fe19bad4bf1") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 22:07:40.995830 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:40.995725 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:40.995998 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:40.995839 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:40.995998 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:40.995935 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:40.995998 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:40.995958 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7k26" podUID="f805f34e-7fc9-430b-9003-16d9765d6657" Oct 30 22:07:40.996186 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:40.996004 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmkb2" podUID="d3d3d52b-f209-4ad4-b531-4fe19bad4bf1" Oct 30 22:07:40.996186 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:40.996084 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsvm9" podUID="bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9" Oct 30 22:07:41.323120 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:41.323030 2579 generic.go:356] "Generic (PLEG): container finished" podID="b69c485c-88c0-43c5-a69c-d88c0f81fa22" containerID="d6d18042c05990ff20ed562345e676c46c8cf70448d3751e31f2624c129f9c8f" exitCode=0 Oct 30 22:07:41.323120 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:41.323085 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lzh8c" event={"ID":"b69c485c-88c0-43c5-a69c-d88c0f81fa22","Type":"ContainerDied","Data":"d6d18042c05990ff20ed562345e676c46c8cf70448d3751e31f2624c129f9c8f"} Oct 30 22:07:41.325940 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:41.325921 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:07:41.326344 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:41.326319 2579 generic.go:356] "Generic (PLEG): container finished" podID="95637308-9cdc-4a32-b573-70e1b4cff546" containerID="fd271464d9e5a6c0303c47684099968b3bf6a61bf4fc98388a40041007d71b3a" exitCode=1 Oct 30 22:07:41.326397 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:41.326380 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" event={"ID":"95637308-9cdc-4a32-b573-70e1b4cff546","Type":"ContainerStarted","Data":"f343d0d6793efe08ca332921b95d51248ddb66dd322484321de1cfa74e59f94b"} Oct 30 22:07:41.326451 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:41.326440 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" event={"ID":"95637308-9cdc-4a32-b573-70e1b4cff546","Type":"ContainerStarted","Data":"830b5867570c91d49c04ff1d27b30e0b0b8bfcd9262e8cf5a94f083119854a93"} Oct 30 22:07:41.326500 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:41.326475 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" event={"ID":"95637308-9cdc-4a32-b573-70e1b4cff546","Type":"ContainerStarted","Data":"21cd1f9cc487d07dc200d66cbc0b4f00e6cebe2958ded023c4c9b78fdf5198dd"} Oct 30 22:07:41.326531 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:41.326509 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" event={"ID":"95637308-9cdc-4a32-b573-70e1b4cff546","Type":"ContainerStarted","Data":"e76ac6c2b600ea47903e1c312b6b97583332a528ecdbdb8f04d4dfa4b0a2ec58"} Oct 30 22:07:41.326558 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:41.326537 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" event={"ID":"95637308-9cdc-4a32-b573-70e1b4cff546","Type":"ContainerDied","Data":"fd271464d9e5a6c0303c47684099968b3bf6a61bf4fc98388a40041007d71b3a"} Oct 30 22:07:41.326596 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:41.326575 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" event={"ID":"95637308-9cdc-4a32-b573-70e1b4cff546","Type":"ContainerStarted","Data":"1af936fdde61d3dc52f1c61b9dcbc02ba33ff369574be711a9cc00d0a982d9d0"} Oct 30 22:07:42.331871 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:42.331839 2579 generic.go:356] "Generic (PLEG): container finished" podID="b69c485c-88c0-43c5-a69c-d88c0f81fa22" containerID="bce12887bfc462cf9745baee6f60759ab78522159f86d263eb3cbd65704b3a76" exitCode=0 Oct 30 22:07:42.332414 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:42.331892 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lzh8c" event={"ID":"b69c485c-88c0-43c5-a69c-d88c0f81fa22","Type":"ContainerDied","Data":"bce12887bfc462cf9745baee6f60759ab78522159f86d263eb3cbd65704b3a76"} Oct 30 22:07:42.996185 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:42.996148 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:42.996185 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:42.996170 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:42.996441 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:42.996263 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:42.996441 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:42.996283 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmkb2" podUID="d3d3d52b-f209-4ad4-b531-4fe19bad4bf1" Oct 30 22:07:42.996441 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:42.996339 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsvm9" podUID="bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9" Oct 30 22:07:42.996441 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:42.996395 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7k26" podUID="f805f34e-7fc9-430b-9003-16d9765d6657" Oct 30 22:07:43.337014 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:43.336913 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lzh8c" event={"ID":"b69c485c-88c0-43c5-a69c-d88c0f81fa22","Type":"ContainerStarted","Data":"b68c4820b20dbed2b0c303a345156ee8a9a3172700625b110f654356055af913"} Oct 30 22:07:43.339695 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:43.339668 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:07:43.342494 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:43.340323 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" event={"ID":"95637308-9cdc-4a32-b573-70e1b4cff546","Type":"ContainerStarted","Data":"4cb96a710560f43282facd613028bc19a6fc0c97d44ee22d16997ce9b550b2b9"} Oct 30 22:07:43.402908 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:43.402852 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lzh8c" podStartSLOduration=8.169286806 podStartE2EDuration="19.40283806s" totalImagesPullingTime="11.233551254s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:43.389972165 +0000 UTC m=+19.883042960" watchObservedRunningTime="2026-10-30 22:07:43.40283806 +0000 UTC m=+19.895908856" Oct 30 22:07:44.996296 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:44.996259 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:44.996963 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:44.996269 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:44.996963 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:44.996277 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:44.996963 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:44.996465 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmkb2" podUID="d3d3d52b-f209-4ad4-b531-4fe19bad4bf1" Oct 30 22:07:44.996963 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:44.996438 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsvm9" podUID="bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9" Oct 30 22:07:44.996963 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:44.996520 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7k26" podUID="f805f34e-7fc9-430b-9003-16d9765d6657" Oct 30 22:07:45.348399 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:45.348138 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:07:45.348658 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:45.348636 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" event={"ID":"95637308-9cdc-4a32-b573-70e1b4cff546","Type":"ContainerStarted","Data":"d56c771bbbe249e5bbd144bba291de26b25954c213bef73a246546217c4d268f"} Oct 30 22:07:45.348907 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:45.348879 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:45.349018 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:45.348951 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:45.349083 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:45.349037 2579 scope.go:122] "RemoveContainer" containerID="fd271464d9e5a6c0303c47684099968b3bf6a61bf4fc98388a40041007d71b3a" Oct 30 22:07:45.366001 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:45.365976 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:45.366420 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:45.366394 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:46.354609 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:46.354576 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:07:46.355190 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:46.355107 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" event={"ID":"95637308-9cdc-4a32-b573-70e1b4cff546","Type":"ContainerStarted","Data":"bf14377eb25277855bd6476942df8e2a0a1bf53473e6603fac58b2dbe6304df0"} Oct 30 22:07:46.355328 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:46.355314 2579 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 22:07:46.438488 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:46.438433 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" podStartSLOduration=7.347430171 podStartE2EDuration="22.438417475s" totalImagesPullingTime="15.090987304s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:46.425873738 +0000 UTC m=+22.918944534" watchObservedRunningTime="2026-10-30 22:07:46.438417475 +0000 UTC m=+22.931488270" Oct 30 22:07:46.995872 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:46.995842 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:46.996080 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:46.995976 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:46.996080 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:46.995996 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:46.996080 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:46.995999 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmkb2" podUID="d3d3d52b-f209-4ad4-b531-4fe19bad4bf1" Oct 30 22:07:46.996080 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:46.996076 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsvm9" podUID="bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9" Oct 30 22:07:46.996283 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:46.996161 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7k26" podUID="f805f34e-7fc9-430b-9003-16d9765d6657" Oct 30 22:07:47.095122 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:47.095035 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gsvm9"] Oct 30 22:07:47.097793 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:47.097767 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pmkb2"] Oct 30 22:07:47.100848 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:47.100821 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l7k26"] Oct 30 22:07:47.331722 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:47.331688 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:47.357493 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:47.357416 2579 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 22:07:47.357493 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:47.357440 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:47.357493 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:47.357423 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:47.358080 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:47.357421 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:47.358080 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:47.357805 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsvm9" podUID="bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9" Oct 30 22:07:47.358080 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:47.357956 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7k26" podUID="f805f34e-7fc9-430b-9003-16d9765d6657" Oct 30 22:07:47.358080 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:47.358067 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmkb2" podUID="d3d3d52b-f209-4ad4-b531-4fe19bad4bf1" Oct 30 22:07:47.522690 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:47.522655 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:47.986564 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:47.986535 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:48.102493 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:48.102462 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:48.995936 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:48.995900 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:48.996346 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:48.995975 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:48.996346 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:48.996094 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:48.996346 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:48.996095 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7k26" podUID="f805f34e-7fc9-430b-9003-16d9765d6657" Oct 30 22:07:48.996346 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:48.996193 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsvm9" podUID="bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9" Oct 30 22:07:48.996346 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:48.996263 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmkb2" podUID="d3d3d52b-f209-4ad4-b531-4fe19bad4bf1" Oct 30 22:07:49.563863 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:49.563832 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:07:49.564110 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:49.564094 2579 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 22:07:49.574606 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:49.574561 2579 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" podUID="95637308-9cdc-4a32-b573-70e1b4cff546" containerName="ovnkube-controller" probeResult="failure" output="" Oct 30 22:07:49.582994 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:49.582960 2579 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" podUID="95637308-9cdc-4a32-b573-70e1b4cff546" containerName="ovnkube-controller" probeResult="failure" output="" Oct 30 22:07:50.995403 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:50.995368 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:50.995844 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:50.995368 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:50.995844 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:50.995487 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7k26" podUID="f805f34e-7fc9-430b-9003-16d9765d6657" Oct 30 22:07:50.995844 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:50.995573 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsvm9" podUID="bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9" Oct 30 22:07:50.995844 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:50.995368 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:50.995844 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:50.995687 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmkb2" podUID="d3d3d52b-f209-4ad4-b531-4fe19bad4bf1" Oct 30 22:07:51.318652 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:51.318570 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:51.425667 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:51.425637 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:52.359775 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:52.359732 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:52.465488 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:52.465451 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:52.995995 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:52.995743 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:52.996156 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:52.995764 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:52.996156 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:52.996031 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gsvm9" podUID="bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9" Oct 30 22:07:52.996156 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:52.996114 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-l7k26" podUID="f805f34e-7fc9-430b-9003-16d9765d6657" Oct 30 22:07:52.996156 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:52.995795 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:52.996310 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:52.996184 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pmkb2" podUID="d3d3d52b-f209-4ad4-b531-4fe19bad4bf1" Oct 30 22:07:53.304252 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.304168 2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeReady" Oct 30 22:07:53.304419 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.304350 2579 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Oct 30 22:07:53.360479 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.360445 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xtbrk"] Oct 30 22:07:53.364881 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.364842 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:53.366881 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.366865 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-khpnv"] Oct 30 22:07:53.366994 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.366977 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xtbrk" Oct 30 22:07:53.369232 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.369205 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-khpnv" Oct 30 22:07:53.385474 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.384174 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 22:07:53.385474 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.384303 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 22:07:53.385474 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.384348 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 22:07:53.385474 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.385408 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xtbrk"] Oct 30 22:07:53.395871 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.395842 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 22:07:53.395871 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.395868 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-c4t6x\"" Oct 30 22:07:53.396098 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.395843 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-zf4xw\"" Oct 30 22:07:53.396098 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.395929 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 22:07:53.411191 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.411159 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-khpnv"] Oct 30 22:07:53.485303 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.485268 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:53.494343 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.494309 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bf28a7e-a8ce-4f33-923a-5d1539978790-metrics-tls\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:07:53.494343 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.494343 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6scx6\" (UniqueName: \"kubernetes.io/projected/3bf28a7e-a8ce-4f33-923a-5d1539978790-kube-api-access-6scx6\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:07:53.494515 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.494441 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pxnh\" (UniqueName: \"kubernetes.io/projected/24e7e5b3-a4ea-4322-915b-5694ea1f3112-kube-api-access-4pxnh\") pod \"ingress-canary-khpnv\" (UID: \"24e7e5b3-a4ea-4322-915b-5694ea1f3112\") " pod="openshift-ingress-canary/ingress-canary-khpnv" Oct 30 22:07:53.494515 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.494501 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24e7e5b3-a4ea-4322-915b-5694ea1f3112-cert\") pod \"ingress-canary-khpnv\" (UID: \"24e7e5b3-a4ea-4322-915b-5694ea1f3112\") " pod="openshift-ingress-canary/ingress-canary-khpnv" Oct 30 22:07:53.494582 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.494547 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3bf28a7e-a8ce-4f33-923a-5d1539978790-config-volume\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:07:53.494617 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.494585 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bf28a7e-a8ce-4f33-923a-5d1539978790-tmp-dir\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:07:53.595696 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.595611 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pxnh\" (UniqueName: \"kubernetes.io/projected/24e7e5b3-a4ea-4322-915b-5694ea1f3112-kube-api-access-4pxnh\") pod \"ingress-canary-khpnv\" (UID: \"24e7e5b3-a4ea-4322-915b-5694ea1f3112\") " pod="openshift-ingress-canary/ingress-canary-khpnv" Oct 30 22:07:53.595696 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.595678 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24e7e5b3-a4ea-4322-915b-5694ea1f3112-cert\") pod \"ingress-canary-khpnv\" (UID: \"24e7e5b3-a4ea-4322-915b-5694ea1f3112\") " pod="openshift-ingress-canary/ingress-canary-khpnv" Oct 30 22:07:53.595926 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.595702 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3bf28a7e-a8ce-4f33-923a-5d1539978790-config-volume\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:07:53.595926 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.595750 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bf28a7e-a8ce-4f33-923a-5d1539978790-tmp-dir\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:07:53.595926 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.595778 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bf28a7e-a8ce-4f33-923a-5d1539978790-metrics-tls\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:07:53.595926 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.595800 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6scx6\" (UniqueName: \"kubernetes.io/projected/3bf28a7e-a8ce-4f33-923a-5d1539978790-kube-api-access-6scx6\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:07:53.595926 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:53.595895 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 22:07:53.596170 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:53.595986 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bf28a7e-a8ce-4f33-923a-5d1539978790-metrics-tls podName:3bf28a7e-a8ce-4f33-923a-5d1539978790 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:54.095963043 +0000 UTC m=+30.589033830 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3bf28a7e-a8ce-4f33-923a-5d1539978790-metrics-tls") pod "dns-default-xtbrk" (UID: "3bf28a7e-a8ce-4f33-923a-5d1539978790") : secret "dns-default-metrics-tls" not found Oct 30 22:07:53.596366 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.596347 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bf28a7e-a8ce-4f33-923a-5d1539978790-tmp-dir\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:07:53.596559 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.596538 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3bf28a7e-a8ce-4f33-923a-5d1539978790-config-volume\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:07:53.599972 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.599946 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24e7e5b3-a4ea-4322-915b-5694ea1f3112-cert\") pod \"ingress-canary-khpnv\" (UID: \"24e7e5b3-a4ea-4322-915b-5694ea1f3112\") " pod="openshift-ingress-canary/ingress-canary-khpnv" Oct 30 22:07:53.615598 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.615560 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6scx6\" (UniqueName: \"kubernetes.io/projected/3bf28a7e-a8ce-4f33-923a-5d1539978790-kube-api-access-6scx6\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:07:53.615598 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.615578 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pxnh\" (UniqueName: \"kubernetes.io/projected/24e7e5b3-a4ea-4322-915b-5694ea1f3112-kube-api-access-4pxnh\") pod \"ingress-canary-khpnv\" (UID: \"24e7e5b3-a4ea-4322-915b-5694ea1f3112\") " pod="openshift-ingress-canary/ingress-canary-khpnv" Oct 30 22:07:53.683711 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:53.683669 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-khpnv" Oct 30 22:07:54.045278 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.045198 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gs6ts"] Oct 30 22:07:54.061557 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.061319 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-khpnv"] Oct 30 22:07:54.061637 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.061623 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:07:54.071341 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.071317 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gs6ts"] Oct 30 22:07:54.091274 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.091238 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 22:07:54.091428 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.091301 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-l4sc9\"" Oct 30 22:07:54.099015 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.098987 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bf28a7e-a8ce-4f33-923a-5d1539978790-metrics-tls\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:07:54.099175 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:54.099131 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 22:07:54.099230 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:54.099209 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bf28a7e-a8ce-4f33-923a-5d1539978790-metrics-tls podName:3bf28a7e-a8ce-4f33-923a-5d1539978790 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:55.099184472 +0000 UTC m=+31.592255269 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3bf28a7e-a8ce-4f33-923a-5d1539978790-metrics-tls") pod "dns-default-xtbrk" (UID: "3bf28a7e-a8ce-4f33-923a-5d1539978790") : secret "dns-default-metrics-tls" not found Oct 30 22:07:54.111770 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.111738 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-r9szz-8wv8c"] Oct 30 22:07:54.120837 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.120809 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:07:54.133456 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.133430 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-r9szz-8wv8c"] Oct 30 22:07:54.135512 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.135487 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:54.140897 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.140874 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 22:07:54.141039 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.140914 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Oct 30 22:07:54.141147 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.141100 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 22:07:54.141147 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.141108 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 22:07:54.156702 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.156679 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Oct 30 22:07:54.191111 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.191087 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-m5kkj\"" Oct 30 22:07:54.199326 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.199296 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/04f40ed9-0eef-4946-b3e6-feafcfee8e03-archives-path\") pod \"periodic-gathering-r9szz-8wv8c\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:07:54.199456 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.199332 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/780de4be-6a4c-4afa-9fee-a3391101d004-crio-socket\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:07:54.199523 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.199487 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/780de4be-6a4c-4afa-9fee-a3391101d004-data-volume\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:07:54.199573 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.199523 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/780de4be-6a4c-4afa-9fee-a3391101d004-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:07:54.199573 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.199555 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:07:54.199656 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.199586 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szj9w\" (UniqueName: \"kubernetes.io/projected/04f40ed9-0eef-4946-b3e6-feafcfee8e03-kube-api-access-szj9w\") pod \"periodic-gathering-r9szz-8wv8c\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:07:54.199656 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.199633 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04f40ed9-0eef-4946-b3e6-feafcfee8e03-service-ca-bundle\") pod \"periodic-gathering-r9szz-8wv8c\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:07:54.199771 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.199659 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgtzk\" (UniqueName: \"kubernetes.io/projected/780de4be-6a4c-4afa-9fee-a3391101d004-kube-api-access-xgtzk\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:07:54.199771 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.199676 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04f40ed9-0eef-4946-b3e6-feafcfee8e03-trusted-ca-bundle\") pod \"periodic-gathering-r9szz-8wv8c\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:07:54.273202 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.273170 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-lc6lf"] Oct 30 22:07:54.285350 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.285324 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:54.290029 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.290005 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-lc6lf" Oct 30 22:07:54.298513 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.298484 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-lc6lf"] Oct 30 22:07:54.300309 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.300286 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgtzk\" (UniqueName: \"kubernetes.io/projected/780de4be-6a4c-4afa-9fee-a3391101d004-kube-api-access-xgtzk\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:07:54.300399 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.300317 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04f40ed9-0eef-4946-b3e6-feafcfee8e03-trusted-ca-bundle\") pod \"periodic-gathering-r9szz-8wv8c\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:07:54.300399 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.300353 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/04f40ed9-0eef-4946-b3e6-feafcfee8e03-archives-path\") pod \"periodic-gathering-r9szz-8wv8c\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:07:54.300399 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.300372 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/780de4be-6a4c-4afa-9fee-a3391101d004-crio-socket\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:07:54.300579 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.300519 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/780de4be-6a4c-4afa-9fee-a3391101d004-crio-socket\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:07:54.300579 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.300536 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/780de4be-6a4c-4afa-9fee-a3391101d004-data-volume\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:07:54.300685 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.300580 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/780de4be-6a4c-4afa-9fee-a3391101d004-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:07:54.300685 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.300617 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:07:54.300785 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:54.300707 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 22:07:54.300785 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:54.300730 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 22:07:54.300785 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.300739 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/04f40ed9-0eef-4946-b3e6-feafcfee8e03-archives-path\") pod \"periodic-gathering-r9szz-8wv8c\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:07:54.300903 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:54.300799 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/780de4be-6a4c-4afa-9fee-a3391101d004-kube-rbac-proxy-cm podName:780de4be-6a4c-4afa-9fee-a3391101d004 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:54.800759137 +0000 UTC m=+31.293829926 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/780de4be-6a4c-4afa-9fee-a3391101d004-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gs6ts" (UID: "780de4be-6a4c-4afa-9fee-a3391101d004") : configmap "kube-rbac-proxy" not found Oct 30 22:07:54.300903 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:54.300821 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls podName:780de4be-6a4c-4afa-9fee-a3391101d004 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:54.800811223 +0000 UTC m=+31.293882004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gs6ts" (UID: "780de4be-6a4c-4afa-9fee-a3391101d004") : secret "insights-runtime-extractor-tls" not found Oct 30 22:07:54.300903 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.300841 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szj9w\" (UniqueName: \"kubernetes.io/projected/04f40ed9-0eef-4946-b3e6-feafcfee8e03-kube-api-access-szj9w\") pod \"periodic-gathering-r9szz-8wv8c\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:07:54.300903 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.300889 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04f40ed9-0eef-4946-b3e6-feafcfee8e03-service-ca-bundle\") pod \"periodic-gathering-r9szz-8wv8c\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:07:54.301132 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.300902 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/780de4be-6a4c-4afa-9fee-a3391101d004-data-volume\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:07:54.301413 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.301386 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04f40ed9-0eef-4946-b3e6-feafcfee8e03-service-ca-bundle\") pod \"periodic-gathering-r9szz-8wv8c\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:07:54.301521 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.301422 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04f40ed9-0eef-4946-b3e6-feafcfee8e03-trusted-ca-bundle\") pod \"periodic-gathering-r9szz-8wv8c\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:07:54.309561 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.309537 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Oct 30 22:07:54.309671 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.309596 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Oct 30 22:07:54.309671 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.309645 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-fr6b8\"" Oct 30 22:07:54.327163 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.327134 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgtzk\" (UniqueName: \"kubernetes.io/projected/780de4be-6a4c-4afa-9fee-a3391101d004-kube-api-access-xgtzk\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:07:54.327309 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.327295 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szj9w\" (UniqueName: \"kubernetes.io/projected/04f40ed9-0eef-4946-b3e6-feafcfee8e03-kube-api-access-szj9w\") pod \"periodic-gathering-r9szz-8wv8c\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:07:54.374735 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.374693 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-khpnv" event={"ID":"24e7e5b3-a4ea-4322-915b-5694ea1f3112","Type":"ContainerStarted","Data":"aa0ca7df9f08416cf33290a4e027b4c09360ae7eead13aebed662755bcdf2607"} Oct 30 22:07:54.402021 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.401986 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwrm9\" (UniqueName: \"kubernetes.io/projected/c1f53f6e-aadc-41fd-aa64-20a466fb1769-kube-api-access-kwrm9\") pod \"migrator-66df5d496b-lc6lf\" (UID: \"c1f53f6e-aadc-41fd-aa64-20a466fb1769\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-lc6lf" Oct 30 22:07:54.404105 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.404078 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:54.441163 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.441129 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:07:54.503896 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.503867 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwrm9\" (UniqueName: \"kubernetes.io/projected/c1f53f6e-aadc-41fd-aa64-20a466fb1769-kube-api-access-kwrm9\") pod \"migrator-66df5d496b-lc6lf\" (UID: \"c1f53f6e-aadc-41fd-aa64-20a466fb1769\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-lc6lf" Oct 30 22:07:54.523461 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.523424 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:54.526385 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.526328 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwrm9\" (UniqueName: \"kubernetes.io/projected/c1f53f6e-aadc-41fd-aa64-20a466fb1769-kube-api-access-kwrm9\") pod \"migrator-66df5d496b-lc6lf\" (UID: \"c1f53f6e-aadc-41fd-aa64-20a466fb1769\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-lc6lf" Oct 30 22:07:54.616626 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.616544 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-lc6lf" Oct 30 22:07:54.635750 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.635717 2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-69qgk"] Oct 30 22:07:54.647762 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.647733 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-69qgk" Oct 30 22:07:54.658505 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.658476 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-69qgk"] Oct 30 22:07:54.665034 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.664090 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Oct 30 22:07:54.665034 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.664247 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Oct 30 22:07:54.665034 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.664325 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Oct 30 22:07:54.665034 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.664443 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-f8wq8\"" Oct 30 22:07:54.665034 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.664500 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Oct 30 22:07:54.747456 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.747423 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-r9szz-8wv8c"] Oct 30 22:07:54.751537 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:54.751505 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04f40ed9_0eef_4946_b3e6_feafcfee8e03.slice/crio-599f89938d5df802596ebac1c4370ccd7550633902cc5876c877b1193910f301 WatchSource:0}: Error finding container 599f89938d5df802596ebac1c4370ccd7550633902cc5876c877b1193910f301: Status 404 returned error can't find the container with id 599f89938d5df802596ebac1c4370ccd7550633902cc5876c877b1193910f301 Oct 30 22:07:54.806141 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.806109 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l28gx\" (UniqueName: \"kubernetes.io/projected/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4-kube-api-access-l28gx\") pod \"service-ca-5fffc97bdf-69qgk\" (UID: \"d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4\") " pod="openshift-service-ca/service-ca-5fffc97bdf-69qgk" Oct 30 22:07:54.806327 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.806260 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/780de4be-6a4c-4afa-9fee-a3391101d004-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:07:54.806378 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:54.806334 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 22:07:54.806417 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:54.806388 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/780de4be-6a4c-4afa-9fee-a3391101d004-kube-rbac-proxy-cm podName:780de4be-6a4c-4afa-9fee-a3391101d004 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:55.806370232 +0000 UTC m=+32.299441020 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/780de4be-6a4c-4afa-9fee-a3391101d004-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gs6ts" (UID: "780de4be-6a4c-4afa-9fee-a3391101d004") : configmap "kube-rbac-proxy" not found Oct 30 22:07:54.806716 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.806700 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:07:54.806785 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.806753 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4-signing-key\") pod \"service-ca-5fffc97bdf-69qgk\" (UID: \"d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4\") " pod="openshift-service-ca/service-ca-5fffc97bdf-69qgk" Oct 30 22:07:54.806825 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.806810 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4-signing-cabundle\") pod \"service-ca-5fffc97bdf-69qgk\" (UID: \"d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4\") " pod="openshift-service-ca/service-ca-5fffc97bdf-69qgk" Oct 30 22:07:54.806930 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:54.806918 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 22:07:54.806971 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:54.806961 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls podName:780de4be-6a4c-4afa-9fee-a3391101d004 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:55.806947199 +0000 UTC m=+32.300017987 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gs6ts" (UID: "780de4be-6a4c-4afa-9fee-a3391101d004") : secret "insights-runtime-extractor-tls" not found Oct 30 22:07:54.907614 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.907535 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4-signing-key\") pod \"service-ca-5fffc97bdf-69qgk\" (UID: \"d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4\") " pod="openshift-service-ca/service-ca-5fffc97bdf-69qgk" Oct 30 22:07:54.907773 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.907628 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4-signing-cabundle\") pod \"service-ca-5fffc97bdf-69qgk\" (UID: \"d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4\") " pod="openshift-service-ca/service-ca-5fffc97bdf-69qgk" Oct 30 22:07:54.907773 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.907671 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l28gx\" (UniqueName: \"kubernetes.io/projected/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4-kube-api-access-l28gx\") pod \"service-ca-5fffc97bdf-69qgk\" (UID: \"d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4\") " pod="openshift-service-ca/service-ca-5fffc97bdf-69qgk" Oct 30 22:07:54.908912 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.908879 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4-signing-cabundle\") pod \"service-ca-5fffc97bdf-69qgk\" (UID: \"d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4\") " pod="openshift-service-ca/service-ca-5fffc97bdf-69qgk" Oct 30 22:07:54.913498 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.913472 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4-signing-key\") pod \"service-ca-5fffc97bdf-69qgk\" (UID: \"d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4\") " pod="openshift-service-ca/service-ca-5fffc97bdf-69qgk" Oct 30 22:07:54.929855 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.929820 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l28gx\" (UniqueName: \"kubernetes.io/projected/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4-kube-api-access-l28gx\") pod \"service-ca-5fffc97bdf-69qgk\" (UID: \"d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4\") " pod="openshift-service-ca/service-ca-5fffc97bdf-69qgk" Oct 30 22:07:54.948322 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.948299 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-lc6lf"] Oct 30 22:07:54.950345 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:54.950320 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1f53f6e_aadc_41fd_aa64_20a466fb1769.slice/crio-0a00291d9a933cb4a78a5712b9ece77fd2f46582b2983f5acaba2155f9646c27 WatchSource:0}: Error finding container 0a00291d9a933cb4a78a5712b9ece77fd2f46582b2983f5acaba2155f9646c27: Status 404 returned error can't find the container with id 0a00291d9a933cb4a78a5712b9ece77fd2f46582b2983f5acaba2155f9646c27 Oct 30 22:07:54.979148 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.979108 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-69qgk" Oct 30 22:07:54.996324 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.996294 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:54.996324 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.996322 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:54.996552 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:54.996306 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:55.027258 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:55.010294 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 22:07:55.027258 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:55.010529 2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 22:07:55.027258 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:55.010969 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 22:07:55.027258 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:55.011102 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-g99sl\"" Oct 30 22:07:55.027258 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:55.011201 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-v75tb\"" Oct 30 22:07:55.027258 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:55.011258 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5n25k\"" Oct 30 22:07:55.027258 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:55.011317 2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 22:07:55.110561 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:55.110527 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bf28a7e-a8ce-4f33-923a-5d1539978790-metrics-tls\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:07:55.110732 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:55.110710 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 22:07:55.110801 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:55.110790 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bf28a7e-a8ce-4f33-923a-5d1539978790-metrics-tls podName:3bf28a7e-a8ce-4f33-923a-5d1539978790 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:57.110767876 +0000 UTC m=+33.603838666 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3bf28a7e-a8ce-4f33-923a-5d1539978790-metrics-tls") pod "dns-default-xtbrk" (UID: "3bf28a7e-a8ce-4f33-923a-5d1539978790") : secret "dns-default-metrics-tls" not found Oct 30 22:07:55.140690 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:55.140660 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:55.256864 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:55.256759 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:55.320998 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:55.320909 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-69qgk"] Oct 30 22:07:55.323241 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:55.323211 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6a0f8e7_7e6d_46d2_97bc_94ce55deb6d4.slice/crio-458725c86296567a240d35bfb92bb7b3f3b5696cd8f028ef030553c57811e4e1 WatchSource:0}: Error finding container 458725c86296567a240d35bfb92bb7b3f3b5696cd8f028ef030553c57811e4e1: Status 404 returned error can't find the container with id 458725c86296567a240d35bfb92bb7b3f3b5696cd8f028ef030553c57811e4e1 Oct 30 22:07:55.378509 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:55.378477 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-69qgk" event={"ID":"d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4","Type":"ContainerStarted","Data":"458725c86296567a240d35bfb92bb7b3f3b5696cd8f028ef030553c57811e4e1"} Oct 30 22:07:55.379927 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:55.379898 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-lc6lf" event={"ID":"c1f53f6e-aadc-41fd-aa64-20a466fb1769","Type":"ContainerStarted","Data":"0a00291d9a933cb4a78a5712b9ece77fd2f46582b2983f5acaba2155f9646c27"} Oct 30 22:07:55.381363 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:55.381339 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-r9szz-8wv8c" event={"ID":"04f40ed9-0eef-4946-b3e6-feafcfee8e03","Type":"ContainerStarted","Data":"599f89938d5df802596ebac1c4370ccd7550633902cc5876c877b1193910f301"} Oct 30 22:07:55.549983 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:55.549902 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:55.816168 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:55.816075 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/780de4be-6a4c-4afa-9fee-a3391101d004-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:07:55.816168 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:55.816121 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:07:55.816389 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:55.816260 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 22:07:55.816389 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:55.816324 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls podName:780de4be-6a4c-4afa-9fee-a3391101d004 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:57.816306782 +0000 UTC m=+34.309377556 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gs6ts" (UID: "780de4be-6a4c-4afa-9fee-a3391101d004") : secret "insights-runtime-extractor-tls" not found Oct 30 22:07:55.816788 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:55.816710 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 22:07:55.816788 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:55.816759 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/780de4be-6a4c-4afa-9fee-a3391101d004-kube-rbac-proxy-cm podName:780de4be-6a4c-4afa-9fee-a3391101d004 nodeName:}" failed. No retries permitted until 2026-10-30 22:07:57.816748971 +0000 UTC m=+34.309819745 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/780de4be-6a4c-4afa-9fee-a3391101d004-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gs6ts" (UID: "780de4be-6a4c-4afa-9fee-a3391101d004") : configmap "kube-rbac-proxy" not found Oct 30 22:07:56.474219 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:56.474185 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:56.581897 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:56.581861 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:56.724793 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:56.724254 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:56.724793 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:56.724344 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs\") pod \"network-metrics-daemon-l7k26\" (UID: \"f805f34e-7fc9-430b-9003-16d9765d6657\") " pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:07:56.724793 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:56.724469 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Oct 30 22:07:56.724793 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:56.724538 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs podName:f805f34e-7fc9-430b-9003-16d9765d6657 nodeName:}" failed. No retries permitted until 2026-10-30 22:08:28.724521491 +0000 UTC m=+65.217592266 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs") pod "network-metrics-daemon-l7k26" (UID: "f805f34e-7fc9-430b-9003-16d9765d6657") : secret "metrics-daemon-secret" not found Oct 30 22:07:56.726855 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:56.726831 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:56.825809 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:56.825286 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wrvl\" (UniqueName: \"kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl\") pod \"network-check-target-pmkb2\" (UID: \"d3d3d52b-f209-4ad4-b531-4fe19bad4bf1\") " pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:56.829012 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:56.828989 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wrvl\" (UniqueName: \"kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl\") pod \"network-check-target-pmkb2\" (UID: \"d3d3d52b-f209-4ad4-b531-4fe19bad4bf1\") " pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:56.848488 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:56.848463 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:07:56.897355 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:56.897174 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:07:57.130033 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:57.129999 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bf28a7e-a8ce-4f33-923a-5d1539978790-metrics-tls\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:07:57.130241 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:57.130180 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 22:07:57.130292 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:57.130241 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bf28a7e-a8ce-4f33-923a-5d1539978790-metrics-tls podName:3bf28a7e-a8ce-4f33-923a-5d1539978790 nodeName:}" failed. No retries permitted until 2026-10-30 22:08:01.130226406 +0000 UTC m=+37.623297180 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3bf28a7e-a8ce-4f33-923a-5d1539978790-metrics-tls") pod "dns-default-xtbrk" (UID: "3bf28a7e-a8ce-4f33-923a-5d1539978790") : secret "dns-default-metrics-tls" not found Oct 30 22:07:57.165247 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:57.165176 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pmkb2"] Oct 30 22:07:57.224937 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:57.224913 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gsvm9"] Oct 30 22:07:57.274394 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:57.274305 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3d3d52b_f209_4ad4_b531_4fe19bad4bf1.slice/crio-ea5089406e59eccdcc736a3a1029e09e0c23c74a0931d87a42f891f6fba855f7 WatchSource:0}: Error finding container ea5089406e59eccdcc736a3a1029e09e0c23c74a0931d87a42f891f6fba855f7: Status 404 returned error can't find the container with id ea5089406e59eccdcc736a3a1029e09e0c23c74a0931d87a42f891f6fba855f7 Oct 30 22:07:57.275392 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:07:57.275365 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdc1c8ae_38f2_4bb3_be0e_1920a21ee0e9.slice/crio-eba5686aef29561315892d0735e28949556767cacb184a90b0bc0b66c1182104 WatchSource:0}: Error finding container eba5686aef29561315892d0735e28949556767cacb184a90b0bc0b66c1182104: Status 404 returned error can't find the container with id eba5686aef29561315892d0735e28949556767cacb184a90b0bc0b66c1182104 Oct 30 22:07:57.389571 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:57.389532 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pmkb2" event={"ID":"d3d3d52b-f209-4ad4-b531-4fe19bad4bf1","Type":"ContainerStarted","Data":"ea5089406e59eccdcc736a3a1029e09e0c23c74a0931d87a42f891f6fba855f7"} Oct 30 22:07:57.391740 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:57.391711 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-lc6lf" event={"ID":"c1f53f6e-aadc-41fd-aa64-20a466fb1769","Type":"ContainerStarted","Data":"d0c2f63e583b6bc05122c73735691c17019394d28f3c29a9d47194174267b648"} Oct 30 22:07:57.393280 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:57.393243 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gsvm9" event={"ID":"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9","Type":"ContainerStarted","Data":"eba5686aef29561315892d0735e28949556767cacb184a90b0bc0b66c1182104"} Oct 30 22:07:57.430030 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:57.429965 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-lc6lf" podStartSLOduration=1.6290774319999999 podStartE2EDuration="3.429944037s" totalImagesPullingTime="1.800866605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:57.42837985 +0000 UTC m=+33.921450647" watchObservedRunningTime="2026-10-30 22:07:57.429944037 +0000 UTC m=+33.923014831" Oct 30 22:07:57.835736 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:57.835700 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/780de4be-6a4c-4afa-9fee-a3391101d004-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:07:57.835736 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:57.835740 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:07:57.836267 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:57.835890 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 22:07:57.836267 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:07:57.835964 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls podName:780de4be-6a4c-4afa-9fee-a3391101d004 nodeName:}" failed. No retries permitted until 2026-10-30 22:08:01.835943661 +0000 UTC m=+38.329014459 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gs6ts" (UID: "780de4be-6a4c-4afa-9fee-a3391101d004") : secret "insights-runtime-extractor-tls" not found Oct 30 22:07:57.836384 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:57.836355 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/780de4be-6a4c-4afa-9fee-a3391101d004-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:07:58.007455 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:58.007421 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:58.118832 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:58.115864 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:58.398248 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:58.398212 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-r9szz-8wv8c" event={"ID":"04f40ed9-0eef-4946-b3e6-feafcfee8e03","Type":"ContainerStarted","Data":"3c39f13c95a43f2169a67c66d6271cf08a4a4bd9e2cffe02b69c6d5e7d359a8e"} Oct 30 22:07:58.400346 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:58.400294 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-khpnv" event={"ID":"24e7e5b3-a4ea-4322-915b-5694ea1f3112","Type":"ContainerStarted","Data":"d091cabfb5648b797299c33c60518ceadc3f901ebac4c1daf7b44055aa47e22b"} Oct 30 22:07:58.437154 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:58.436275 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-r9szz-8wv8c" podStartSLOduration=1.236936929 podStartE2EDuration="4.436262432s" totalImagesPullingTime="3.199325503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:58.43612982 +0000 UTC m=+34.929200617" watchObservedRunningTime="2026-10-30 22:07:58.436262432 +0000 UTC m=+34.929333228" Oct 30 22:07:58.476815 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:58.475735 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-khpnv" podStartSLOduration=2.22820517 podStartE2EDuration="5.475711786s" totalImagesPullingTime="3.247506616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:58.473860393 +0000 UTC m=+34.966931190" watchObservedRunningTime="2026-10-30 22:07:58.475711786 +0000 UTC m=+34.968782617" Oct 30 22:07:59.339452 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:59.339362 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:59.404984 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:59.404942 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-69qgk" event={"ID":"d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4","Type":"ContainerStarted","Data":"a271d518a33652aa5e602d9e1ac2fdee543123dfa531f6099cefdef17a6fd8d0"} Oct 30 22:07:59.482075 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:59.481788 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:59.617556 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:59.617468 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:07:59.734773 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:07:59.734709 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:00.041617 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:08:00.041547 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:00.642898 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:08:00.642770 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:01.169636 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:08:01.169599 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bf28a7e-a8ce-4f33-923a-5d1539978790-metrics-tls\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:08:01.172125 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:08:01.172098 2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bf28a7e-a8ce-4f33-923a-5d1539978790-metrics-tls\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:08:01.182067 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:08:01.182027 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xtbrk" Oct 30 22:08:01.229234 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:08:01.229200 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:01.430261 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:08:01.430226 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:01.593788 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:08:01.593172 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-69qgk" podStartSLOduration=4.614909375 podStartE2EDuration="7.593149656s" totalImagesPullingTime="2.978240281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:07:59.443593084 +0000 UTC m=+35.936663883" watchObservedRunningTime="2026-10-30 22:08:01.593149656 +0000 UTC m=+38.086220453" Oct 30 22:08:01.593788 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:08:01.593441 2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xtbrk"] Oct 30 22:08:01.595773 ip-10-0-131-94 kubenswrapper[2579]: W1030 22:08:01.595744 2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bf28a7e_a8ce_4f33_923a_5d1539978790.slice/crio-cea00db705ec975de38c10812f6b5f23be096f68d1eca46642c5f2d97c11ac33 WatchSource:0}: Error finding container cea00db705ec975de38c10812f6b5f23be096f68d1eca46642c5f2d97c11ac33: Status 404 returned error can't find the container with id cea00db705ec975de38c10812f6b5f23be096f68d1eca46642c5f2d97c11ac33 Oct 30 22:08:01.676760 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:08:01.676654 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:01.839644 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:08:01.839602 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q5gw7_c135a307-61bc-4e34-9b5c-b97793255db7/dns-node-resolver/0.log" Oct 30 22:08:01.876544 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:08:01.876507 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:08:01.876775 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:08:01.876694 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 22:08:01.876775 ip-10-0-131-94 kubenswrapper[2579]: E1030 22:08:01.876767 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls podName:780de4be-6a4c-4afa-9fee-a3391101d004 nodeName:}" failed. No retries permitted until 2026-10-30 22:08:09.876747911 +0000 UTC m=+46.369818700 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gs6ts" (UID: "780de4be-6a4c-4afa-9fee-a3391101d004") : secret "insights-runtime-extractor-tls" not found Oct 30 22:08:02.035581 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:08:02.035467 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:02.415865 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:08:02.415823 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pmkb2" event={"ID":"d3d3d52b-f209-4ad4-b531-4fe19bad4bf1","Type":"ContainerStarted","Data":"be1cd179c504ad09b2376d85536205aeed66de6931fd1b0eac616db4783f9b09"} Oct 30 22:08:02.416111 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:08:02.416092 2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:08:02.417586 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:08:02.417557 2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xtbrk" event={"ID":"3bf28a7e-a8ce-4f33-923a-5d1539978790","Type":"ContainerStarted","Data":"cea00db705ec975de38c10812f6b5f23be096f68d1eca46642c5f2d97c11ac33"} Oct 30 22:08:02.454731 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:08:02.454671 2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pmkb2" podStartSLOduration=34.441576076 podStartE2EDuration="38.454653999s" totalImagesPullingTime="4.013077923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:02.453946325 +0000 UTC m=+38.947017117" watchObservedRunningTime="2026-10-30 22:08:02.454653999 +0000 UTC m=+38.947724794" Oct 30 22:08:02.661722 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:08:02.661696 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:02.787645 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:08:02.787595 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:02.893692 ip-10-0-131-94 kubenswrapper[2579]: I1030 22:08:02.893657 2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:02.987492 ip-10-0-131-94 systemd[1]: Stopping Kubernetes Kubelet... Oct 30 22:08:03.000710 ip-10-0-131-94 systemd[1]: kubelet.service: Deactivated successfully. Oct 30 22:08:03.000880 ip-10-0-131-94 systemd[1]: Stopped Kubernetes Kubelet. Oct 30 22:08:03.000938 ip-10-0-131-94 systemd[1]: kubelet.service: Consumed 2.551s CPU time. Oct 30 22:08:03.012628 ip-10-0-131-94 systemd[1]: Starting Kubernetes Kubelet... Oct 30 22:08:03.121516 ip-10-0-131-94 kubenswrapper[4615]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 22:08:03.121516 ip-10-0-131-94 kubenswrapper[4615]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 22:08:03.121516 ip-10-0-131-94 kubenswrapper[4615]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 22:08:03.121516 ip-10-0-131-94 kubenswrapper[4615]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 22:08:03.126473 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126355 4615 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:08:03.126473 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126469 4615 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:08:03.126473 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126476 4615 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:08:03.126473 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126481 4615 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:08:03.126473 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126486 4615 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:08:03.126722 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126490 4615 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:08:03.126722 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126495 4615 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:08:03.126722 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126499 4615 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:08:03.126722 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126503 4615 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:08:03.126722 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126507 4615 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:08:03.126722 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126511 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:08:03.126722 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126517 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:08:03.126722 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126522 4615 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:08:03.126722 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126526 4615 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:08:03.126722 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126531 4615 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:08:03.126722 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126535 4615 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:08:03.126722 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126539 4615 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:08:03.126722 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126543 4615 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:08:03.126722 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126547 4615 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:08:03.126722 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126552 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:08:03.126722 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126556 4615 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:08:03.126722 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126560 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:08:03.126722 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126567 4615 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:08:03.126722 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126574 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:08:03.126722 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126591 4615 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:08:03.127412 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126596 4615 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:08:03.127412 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126603 4615 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:08:03.127412 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126612 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:08:03.127412 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126617 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:08:03.127412 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126622 4615 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:08:03.127412 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126626 4615 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:08:03.127412 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126636 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:08:03.127412 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126641 4615 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:08:03.127412 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126646 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:08:03.127412 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126650 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:08:03.127412 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126654 4615 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:08:03.127412 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126658 4615 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:08:03.127412 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126662 4615 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:08:03.127412 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126666 4615 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:08:03.127412 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126670 4615 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:08:03.127412 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126674 4615 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:08:03.127412 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126678 4615 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:08:03.127412 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126682 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:08:03.127412 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126686 4615 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:08:03.128083 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126690 4615 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:08:03.128083 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126695 4615 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:08:03.128083 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126699 4615 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:08:03.128083 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126704 4615 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:08:03.128083 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126708 4615 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:08:03.128083 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126712 4615 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:08:03.128083 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126717 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:08:03.128083 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126721 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:08:03.128083 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126725 4615 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:08:03.128083 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126731 4615 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:08:03.128083 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126736 4615 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:08:03.128083 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126741 4615 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:08:03.128083 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126745 4615 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:08:03.128083 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126749 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:08:03.128083 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126753 4615 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:08:03.128083 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126758 4615 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:08:03.128083 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126762 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:08:03.128083 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126766 4615 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:08:03.128083 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126770 4615 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:08:03.128756 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126774 4615 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:08:03.128756 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126779 4615 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:08:03.128756 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126784 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:08:03.128756 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126788 4615 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:08:03.128756 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126792 4615 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:08:03.128756 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126796 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:08:03.128756 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126801 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:08:03.128756 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126806 4615 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:08:03.128756 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126810 4615 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:08:03.128756 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126814 4615 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:08:03.128756 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126818 4615 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:08:03.128756 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126823 4615 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:08:03.128756 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126828 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:08:03.128756 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126832 4615 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:08:03.128756 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126837 4615 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:08:03.128756 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126841 4615 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:08:03.128756 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126846 4615 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:08:03.128756 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126850 4615 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:08:03.128756 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126854 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:08:03.129409 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126858 4615 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:08:03.129409 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126863 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:08:03.129409 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126867 4615 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:08:03.129409 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126872 4615 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:08:03.129409 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126876 4615 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:08:03.129409 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126880 4615 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:08:03.129409 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126884 4615 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:08:03.129409 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126900 4615 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:08:03.129409 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126905 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:08:03.129409 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126909 4615 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:08:03.129409 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126914 4615 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:08:03.129409 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126918 4615 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:08:03.129409 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126922 4615 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:08:03.129409 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126927 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:08:03.129409 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126932 4615 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:08:03.129409 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126936 4615 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:08:03.129409 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126941 4615 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:08:03.129409 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126944 4615 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:08:03.129409 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126946 4615 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:08:03.129409 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126949 4615 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:08:03.130139 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126952 4615 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:08:03.130139 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126955 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:08:03.130139 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126960 4615 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:08:03.130139 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.126964 4615 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:08:03.130139 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128258 4615 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:08:03.130139 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128269 4615 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:08:03.130139 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128273 4615 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:08:03.130139 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128278 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:08:03.130139 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128283 4615 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:08:03.130139 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128288 4615 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:08:03.130139 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128293 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:08:03.130139 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128298 4615 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:08:03.130139 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128302 4615 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:08:03.130139 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128306 4615 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:08:03.130139 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128311 4615 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:08:03.130139 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128316 4615 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:08:03.130139 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128320 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:08:03.130139 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128324 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:08:03.130139 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128329 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:08:03.130772 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128333 4615 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:08:03.130772 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128337 4615 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:08:03.130772 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128343 4615 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:08:03.130772 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128350 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:08:03.130772 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128355 4615 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:08:03.130772 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128360 4615 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:08:03.130772 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128364 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:08:03.130772 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128368 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:08:03.130772 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128372 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:08:03.130772 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128378 4615 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:08:03.130772 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128382 4615 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:08:03.130772 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128387 4615 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:08:03.130772 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128391 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:08:03.130772 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128395 4615 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:08:03.130772 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128399 4615 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:08:03.130772 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128403 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:08:03.130772 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128408 4615 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:08:03.130772 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128412 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:08:03.130772 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128416 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:08:03.131484 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128420 4615 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:08:03.131484 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128424 4615 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:08:03.131484 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128429 4615 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:08:03.131484 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128433 4615 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:08:03.131484 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128437 4615 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:08:03.131484 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128441 4615 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:08:03.131484 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128445 4615 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:08:03.131484 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128449 4615 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:08:03.131484 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128453 4615 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:08:03.131484 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128459 4615 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:08:03.131484 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128463 4615 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:08:03.131484 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128468 4615 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:08:03.131484 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128472 4615 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:08:03.131484 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128477 4615 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:08:03.131484 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128482 4615 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:08:03.131484 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128486 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:08:03.131484 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128490 4615 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:08:03.131484 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128495 4615 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:08:03.131484 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128499 4615 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:08:03.131484 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128503 4615 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:08:03.132332 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128507 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:08:03.132332 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128511 4615 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:08:03.132332 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128515 4615 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:08:03.132332 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128520 4615 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:08:03.132332 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128525 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:08:03.132332 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128529 4615 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:08:03.132332 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128533 4615 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:08:03.132332 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128538 4615 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:08:03.132332 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128542 4615 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:08:03.132332 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128548 4615 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:08:03.132332 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128554 4615 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:08:03.132332 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128559 4615 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:08:03.132332 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128564 4615 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:08:03.132332 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128568 4615 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:08:03.132332 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128573 4615 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:08:03.132332 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128577 4615 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:08:03.132332 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128582 4615 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:08:03.132332 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128587 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:08:03.132332 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128592 4615 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:08:03.133019 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128596 4615 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:08:03.133019 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128600 4615 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:08:03.133019 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128604 4615 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:08:03.133019 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128610 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:08:03.133019 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128614 4615 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:08:03.133019 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128619 4615 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:08:03.133019 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128623 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:08:03.133019 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128627 4615 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:08:03.133019 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128631 4615 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:08:03.133019 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128635 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:08:03.133019 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128639 4615 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:08:03.133019 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128643 4615 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:08:03.133019 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128647 4615 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:08:03.133019 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128651 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:08:03.133019 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128655 4615 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:08:03.133019 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128659 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:08:03.133019 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128663 4615 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:08:03.133019 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128667 4615 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:08:03.133019 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128671 4615 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:08:03.133019 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128676 4615 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:08:03.133751 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128680 4615 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:08:03.133751 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128684 4615 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:08:03.133751 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128689 4615 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:08:03.133751 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128693 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:08:03.133751 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128697 4615 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:08:03.133751 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128701 4615 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:08:03.133751 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128706 4615 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:08:03.133751 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128712 4615 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:08:03.133751 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128716 4615 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:08:03.133751 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128720 4615 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:08:03.133751 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128724 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:08:03.133751 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128728 4615 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:08:03.133751 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.128732 4615 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:08:03.133751 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129401 4615 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:08:03.133751 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129410 4615 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:08:03.133751 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129416 4615 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:08:03.133751 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129425 4615 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:08:03.133751 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129429 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:08:03.133751 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129432 4615 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:08:03.133751 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129435 4615 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:08:03.134543 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129438 4615 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:08:03.134543 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129442 4615 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:08:03.134543 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129448 4615 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:08:03.134543 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129453 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:08:03.134543 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129458 4615 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:08:03.134543 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129462 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:08:03.134543 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129467 4615 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:08:03.134543 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129471 4615 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:08:03.134543 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129475 4615 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:08:03.134543 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129478 4615 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:08:03.134543 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129481 4615 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:08:03.134543 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129484 4615 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:08:03.134543 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129487 4615 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:08:03.134543 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129490 4615 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:08:03.134543 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129495 4615 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:08:03.134543 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129499 4615 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:08:03.134543 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129513 4615 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:08:03.134543 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129517 4615 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:08:03.134543 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129520 4615 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:08:03.135304 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129524 4615 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:08:03.135304 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129528 4615 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:08:03.135304 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129533 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:08:03.135304 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129538 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:08:03.135304 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129543 4615 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:08:03.135304 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129548 4615 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:08:03.135304 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129551 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:08:03.135304 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129553 4615 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:08:03.135304 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129556 4615 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:08:03.135304 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129562 4615 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:08:03.135304 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129565 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:08:03.135304 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129568 4615 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:08:03.135304 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129571 4615 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:08:03.135304 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129576 4615 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:08:03.135304 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129580 4615 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:08:03.135304 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129584 4615 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:08:03.135304 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129588 4615 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:08:03.135304 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129592 4615 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:08:03.135304 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129595 4615 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:08:03.136293 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129598 4615 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:08:03.136293 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129600 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:08:03.136293 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129603 4615 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:08:03.136293 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129606 4615 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:08:03.136293 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129608 4615 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:08:03.136293 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129611 4615 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:08:03.136293 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129614 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:08:03.136293 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129616 4615 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:08:03.136293 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129619 4615 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:08:03.136293 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129622 4615 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:08:03.136293 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129624 4615 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:08:03.136293 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129627 4615 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:08:03.136293 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129630 4615 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:08:03.136293 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129632 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:08:03.136293 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129634 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:08:03.136293 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129637 4615 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:08:03.136293 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129639 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:08:03.136293 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129642 4615 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:08:03.136293 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129645 4615 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:08:03.136293 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129647 4615 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:08:03.137200 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129650 4615 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:08:03.137200 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129653 4615 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:08:03.137200 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129659 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:08:03.137200 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129673 4615 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:08:03.137200 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129682 4615 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:08:03.137200 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129688 4615 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:08:03.137200 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129693 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:08:03.137200 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129696 4615 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:08:03.137200 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129699 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:08:03.137200 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129702 4615 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:08:03.137200 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129705 4615 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:08:03.137200 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129708 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:08:03.137200 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129710 4615 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:08:03.137200 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129713 4615 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:08:03.137200 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129716 4615 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:08:03.137200 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129718 4615 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:08:03.137200 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129721 4615 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:08:03.137200 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129725 4615 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:08:03.137200 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129729 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:08:03.138075 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129734 4615 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:08:03.138075 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129739 4615 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:08:03.138075 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129744 4615 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:08:03.138075 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129749 4615 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:08:03.138075 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129753 4615 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:08:03.138075 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129758 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:08:03.138075 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129761 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:08:03.138075 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129763 4615 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:08:03.138075 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129766 4615 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:08:03.138075 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129769 4615 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:08:03.138075 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129771 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:08:03.138075 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129774 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:08:03.138075 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129777 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:08:03.138075 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129779 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:08:03.138075 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129783 4615 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:08:03.138075 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129788 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:08:03.138075 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129791 4615 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:08:03.138075 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129800 4615 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:08:03.138075 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129805 4615 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129810 4615 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129815 4615 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.129820 4615 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.129936 4615 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.129948 4615 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.129962 4615 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.129969 4615 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.129976 4615 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.129980 4615 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.129987 4615 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.129993 4615 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.129999 4615 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130003 4615 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130009 4615 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130015 4615 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130020 4615 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130025 4615 flags.go:64] FLAG: --cgroup-root="" Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130030 4615 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130035 4615 flags.go:64] FLAG: --client-ca-file="" Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130039 4615 flags.go:64] FLAG: --cloud-provider="external" Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130057 4615 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130068 4615 flags.go:64] FLAG: --cluster-domain="" Oct 30 22:08:03.138983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130072 4615 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130077 4615 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130082 4615 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130087 4615 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130094 4615 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130105 4615 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130112 4615 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130117 4615 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130122 4615 flags.go:64] FLAG: --contention-profiling="false" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130135 4615 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130140 4615 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130146 4615 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130153 4615 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130160 4615 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130165 4615 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130169 4615 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130174 4615 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130178 4615 flags.go:64] FLAG: --enable-server="true" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130183 4615 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130194 4615 flags.go:64] FLAG: --event-burst="100" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130197 4615 flags.go:64] FLAG: --event-qps="50" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130201 4615 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130205 4615 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130210 4615 flags.go:64] FLAG: --eviction-hard="" Oct 30 22:08:03.140030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130218 4615 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130224 4615 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130229 4615 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130235 4615 flags.go:64] FLAG: --eviction-soft="" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130241 4615 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130246 4615 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130251 4615 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130256 4615 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130259 4615 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130263 4615 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130269 4615 flags.go:64] FLAG: --feature-gates="" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130275 4615 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130281 4615 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130288 4615 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130294 4615 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130297 4615 flags.go:64] FLAG: --healthz-port="10248" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130301 4615 flags.go:64] FLAG: --help="false" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130304 4615 flags.go:64] FLAG: --hostname-override="ip-10-0-131-94.ec2.internal" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130319 4615 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130325 4615 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130328 4615 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130332 4615 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130336 4615 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130339 4615 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 22:08:03.141169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130342 4615 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130345 4615 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130348 4615 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130352 4615 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130358 4615 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130363 4615 flags.go:64] FLAG: --kube-reserved="" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130369 4615 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130374 4615 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130378 4615 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130381 4615 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130384 4615 flags.go:64] FLAG: --lock-file="" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130388 4615 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130393 4615 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130398 4615 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130406 4615 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130409 4615 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130413 4615 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130418 4615 flags.go:64] FLAG: --logging-format="text" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130423 4615 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130429 4615 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130434 4615 flags.go:64] FLAG: --manifest-url="" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130440 4615 flags.go:64] FLAG: --manifest-url-header="" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130446 4615 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130449 4615 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130453 4615 flags.go:64] FLAG: --max-pods="110" Oct 30 22:08:03.142375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130456 4615 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130459 4615 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130472 4615 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130477 4615 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130482 4615 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130487 4615 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130490 4615 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130499 4615 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130503 4615 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130508 4615 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130513 4615 flags.go:64] FLAG: --pod-cidr="" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130518 4615 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130524 4615 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130529 4615 flags.go:64] FLAG: --pods-per-core="0" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130533 4615 flags.go:64] FLAG: --port="10250" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130539 4615 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130544 4615 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-060ca0ed5af39e278" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130549 4615 flags.go:64] FLAG: --qos-reserved="" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130554 4615 flags.go:64] FLAG: --read-only-port="10255" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130559 4615 flags.go:64] FLAG: --register-node="true" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130564 4615 flags.go:64] FLAG: --register-with-taints="" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130570 4615 flags.go:64] FLAG: --registry-burst="10" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130574 4615 flags.go:64] FLAG: --registry-qps="5" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130577 4615 flags.go:64] FLAG: --reserved-cpus="" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130580 4615 flags.go:64] FLAG: --reserved-memory="" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130584 4615 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 22:08:03.143444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130587 4615 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130590 4615 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130597 4615 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130604 4615 flags.go:64] FLAG: --runonce="false" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130607 4615 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130612 4615 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130617 4615 flags.go:64] FLAG: --seccomp-default="false" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130622 4615 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130625 4615 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130636 4615 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130641 4615 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130646 4615 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130651 4615 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130656 4615 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130660 4615 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130666 4615 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130671 4615 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130676 4615 flags.go:64] FLAG: --system-cgroups="" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130681 4615 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130691 4615 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130695 4615 flags.go:64] FLAG: --tls-cert-file="" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130700 4615 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130710 4615 flags.go:64] FLAG: --tls-min-version="" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130714 4615 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130719 4615 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 22:08:03.144597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130723 4615 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 22:08:03.145694 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130728 4615 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 22:08:03.145694 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130733 4615 flags.go:64] FLAG: --v="2" Oct 30 22:08:03.145694 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130740 4615 flags.go:64] FLAG: --version="false" Oct 30 22:08:03.145694 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130748 4615 flags.go:64] FLAG: --vmodule="" Oct 30 22:08:03.145694 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130755 4615 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 22:08:03.145694 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.130760 4615 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 22:08:03.145694 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.130983 4615 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:08:03.145694 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.130991 4615 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:08:03.145694 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.130998 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:08:03.145694 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131003 4615 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:08:03.145694 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131007 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:08:03.145694 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131012 4615 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:08:03.145694 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131016 4615 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:08:03.145694 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131020 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:08:03.145694 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131024 4615 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:08:03.145694 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131029 4615 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:08:03.145694 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131040 4615 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:08:03.145694 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131058 4615 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:08:03.145694 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131063 4615 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:08:03.145694 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131068 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:08:03.145694 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131072 4615 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:08:03.146621 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131077 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:08:03.146621 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131080 4615 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:08:03.146621 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131086 4615 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:08:03.146621 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131092 4615 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:08:03.146621 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131100 4615 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:08:03.146621 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131104 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:08:03.146621 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131109 4615 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:08:03.146621 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131123 4615 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:08:03.146621 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131127 4615 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:08:03.146621 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131131 4615 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:08:03.146621 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131135 4615 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:08:03.146621 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131140 4615 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:08:03.146621 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131145 4615 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:08:03.146621 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131150 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:08:03.146621 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131154 4615 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:08:03.146621 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131158 4615 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:08:03.146621 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131162 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:08:03.146621 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131166 4615 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:08:03.146621 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131170 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:08:03.147415 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131177 4615 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:08:03.147415 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131181 4615 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:08:03.147415 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131186 4615 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:08:03.147415 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131191 4615 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:08:03.147415 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131196 4615 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:08:03.147415 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131201 4615 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:08:03.147415 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131207 4615 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:08:03.147415 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131213 4615 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:08:03.147415 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131219 4615 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:08:03.147415 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131232 4615 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:08:03.147415 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131237 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:08:03.147415 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131241 4615 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:08:03.147415 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131246 4615 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:08:03.147415 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131249 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:08:03.147415 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131252 4615 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:08:03.147415 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131254 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:08:03.147415 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131257 4615 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:08:03.147415 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131260 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:08:03.147415 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131262 4615 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:08:03.147415 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131268 4615 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:08:03.148238 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131272 4615 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:08:03.148238 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131277 4615 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:08:03.148238 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131282 4615 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:08:03.148238 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131287 4615 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:08:03.148238 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131291 4615 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:08:03.148238 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131296 4615 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:08:03.148238 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131301 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:08:03.148238 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131306 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:08:03.148238 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131309 4615 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:08:03.148238 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131311 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:08:03.148238 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131314 4615 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:08:03.148238 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131317 4615 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:08:03.148238 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131321 4615 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:08:03.148238 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131324 4615 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:08:03.148238 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131327 4615 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:08:03.148238 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131329 4615 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:08:03.148238 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131334 4615 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:08:03.148238 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131338 4615 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:08:03.148238 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131344 4615 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:08:03.148238 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131348 4615 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:08:03.149022 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131353 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:08:03.149022 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131357 4615 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:08:03.149022 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131363 4615 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:08:03.149022 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131368 4615 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:08:03.149022 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131371 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:08:03.149022 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131373 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:08:03.149022 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131376 4615 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:08:03.149022 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131379 4615 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:08:03.149022 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131382 4615 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:08:03.149022 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131385 4615 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:08:03.149022 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131388 4615 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:08:03.149022 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131390 4615 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:08:03.149022 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131393 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:08:03.149022 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131396 4615 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:08:03.149022 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131398 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:08:03.149022 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131401 4615 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:08:03.149022 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131404 4615 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:08:03.149022 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131406 4615 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:08:03.149022 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131409 4615 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:08:03.149945 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131411 4615 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:08:03.149945 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131414 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:08:03.149945 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131417 4615 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:08:03.149945 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131419 4615 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:08:03.149945 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131422 4615 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:08:03.149945 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131426 4615 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:08:03.149945 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131429 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:08:03.149945 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131431 4615 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:08:03.149945 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131434 4615 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:08:03.149945 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131436 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:08:03.149945 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131439 4615 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:08:03.149945 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131442 4615 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:08:03.149945 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131444 4615 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:08:03.149945 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.131490 4615 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 22:08:03.149945 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131559 4615 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:08:03.149945 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131568 4615 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:08:03.150612 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131574 4615 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:08:03.150612 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131579 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:08:03.150612 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131583 4615 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:08:03.150612 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131587 4615 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:08:03.150612 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131592 4615 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:08:03.150612 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131597 4615 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:08:03.150612 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131601 4615 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:08:03.150612 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131607 4615 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:08:03.150612 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131611 4615 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:08:03.150612 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131616 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:08:03.150612 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131620 4615 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:08:03.150612 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131625 4615 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:08:03.150612 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131629 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:08:03.150612 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131641 4615 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:08:03.150612 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131646 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:08:03.150612 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131650 4615 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:08:03.150612 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131655 4615 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:08:03.150612 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131660 4615 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:08:03.150612 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131664 4615 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:08:03.150612 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131669 4615 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:08:03.151460 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131674 4615 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:08:03.151460 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131681 4615 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:08:03.151460 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131688 4615 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:08:03.151460 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131693 4615 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:08:03.151460 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131697 4615 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:08:03.151460 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131701 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:08:03.151460 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131705 4615 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:08:03.151460 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131709 4615 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:08:03.151460 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131713 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:08:03.151460 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131721 4615 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:08:03.151460 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131725 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:08:03.151460 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131730 4615 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:08:03.151460 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131734 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:08:03.151460 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131740 4615 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:08:03.151460 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131745 4615 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:08:03.151460 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131749 4615 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:08:03.151460 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131753 4615 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:08:03.151460 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131757 4615 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:08:03.151460 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131761 4615 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:08:03.152216 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131765 4615 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:08:03.152216 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131769 4615 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:08:03.152216 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131773 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:08:03.152216 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131777 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:08:03.152216 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131781 4615 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:08:03.152216 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131784 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:08:03.152216 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131787 4615 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:08:03.152216 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131790 4615 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:08:03.152216 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131793 4615 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:08:03.152216 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131795 4615 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:08:03.152216 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131799 4615 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:08:03.152216 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131802 4615 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:08:03.152216 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131807 4615 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:08:03.152216 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131811 4615 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:08:03.152216 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131815 4615 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:08:03.152216 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131820 4615 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:08:03.152216 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131824 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:08:03.152216 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131826 4615 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:08:03.152216 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131829 4615 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:08:03.152216 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131832 4615 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:08:03.152985 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131834 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:08:03.152985 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131837 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:08:03.152985 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131841 4615 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:08:03.152985 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131844 4615 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:08:03.152985 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131848 4615 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:08:03.152985 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131852 4615 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:08:03.152985 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131857 4615 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:08:03.152985 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131862 4615 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:08:03.152985 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131867 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:08:03.152985 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131872 4615 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:08:03.152985 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131876 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:08:03.152985 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131880 4615 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:08:03.152985 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131884 4615 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:08:03.152985 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131888 4615 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:08:03.152985 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131894 4615 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:08:03.152985 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131898 4615 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:08:03.152985 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131910 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:08:03.152985 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131915 4615 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:08:03.152985 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131918 4615 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:08:03.153784 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131921 4615 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:08:03.153784 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131923 4615 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:08:03.153784 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131926 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:08:03.153784 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131929 4615 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:08:03.153784 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131931 4615 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:08:03.153784 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131934 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:08:03.153784 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131936 4615 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:08:03.153784 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131939 4615 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:08:03.153784 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131942 4615 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:08:03.153784 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131946 4615 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:08:03.153784 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131951 4615 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:08:03.153784 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131955 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:08:03.153784 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131958 4615 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:08:03.153784 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131961 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:08:03.153784 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131964 4615 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:08:03.153784 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131969 4615 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:08:03.153784 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131974 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:08:03.153784 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131979 4615 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:08:03.153784 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131984 4615 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:08:03.154636 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131989 4615 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:08:03.154636 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131993 4615 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:08:03.154636 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.131999 4615 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:08:03.154636 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.132003 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:08:03.154636 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.132007 4615 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:08:03.154636 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.132012 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:08:03.154636 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.132016 4615 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:08:03.154636 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.137773 4615 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 22:08:03.154636 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.137796 4615 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 22:08:03.154636 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.137822 4615 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 22:08:03.154636 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.137830 4615 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 22:08:03.154636 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.137947 4615 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:08:03.154636 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.137956 4615 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:08:03.154636 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.137961 4615 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:08:03.154636 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.137966 4615 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:08:03.154636 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.137971 4615 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:08:03.154636 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.137977 4615 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:08:03.154636 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.137981 4615 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:08:03.154636 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.137986 4615 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:08:03.154636 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.137990 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:08:03.155431 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.137994 4615 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:08:03.155431 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.137998 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:08:03.155431 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138003 4615 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:08:03.155431 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138007 4615 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:08:03.155431 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138011 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:08:03.155431 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138016 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:08:03.155431 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138020 4615 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:08:03.155431 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138024 4615 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:08:03.155431 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138029 4615 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:08:03.155431 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138033 4615 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:08:03.155431 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138037 4615 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:08:03.155431 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138059 4615 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:08:03.155431 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138064 4615 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:08:03.155431 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138068 4615 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:08:03.155431 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138072 4615 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:08:03.155431 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138076 4615 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:08:03.155431 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138080 4615 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:08:03.155431 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138084 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:08:03.155431 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138088 4615 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:08:03.155431 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138092 4615 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:08:03.156188 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138107 4615 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:08:03.156188 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138115 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:08:03.156188 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138121 4615 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:08:03.156188 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138126 4615 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:08:03.156188 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138130 4615 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:08:03.156188 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138135 4615 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:08:03.156188 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138139 4615 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:08:03.156188 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138143 4615 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:08:03.156188 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138148 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:08:03.156188 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138152 4615 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:08:03.156188 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138157 4615 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:08:03.156188 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138161 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:08:03.156188 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138165 4615 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:08:03.156188 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138170 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:08:03.156188 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138174 4615 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:08:03.156188 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138178 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:08:03.156188 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138183 4615 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:08:03.156188 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138187 4615 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:08:03.156188 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138191 4615 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:08:03.156894 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138196 4615 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:08:03.156894 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138200 4615 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:08:03.156894 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138204 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:08:03.156894 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138208 4615 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:08:03.156894 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138212 4615 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:08:03.156894 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138217 4615 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:08:03.156894 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138221 4615 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:08:03.156894 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138225 4615 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:08:03.156894 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138229 4615 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:08:03.156894 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138233 4615 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:08:03.156894 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138238 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:08:03.156894 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138242 4615 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:08:03.156894 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138247 4615 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:08:03.156894 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138252 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:08:03.156894 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138264 4615 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:08:03.156894 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138269 4615 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:08:03.156894 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138273 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:08:03.156894 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138276 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:08:03.156894 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138282 4615 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:08:03.157840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138287 4615 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:08:03.157840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138292 4615 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:08:03.157840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138297 4615 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:08:03.157840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138301 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:08:03.157840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138305 4615 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:08:03.157840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138310 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:08:03.157840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138314 4615 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:08:03.157840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138318 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:08:03.157840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138322 4615 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:08:03.157840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138326 4615 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:08:03.157840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138331 4615 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:08:03.157840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138335 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:08:03.157840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138339 4615 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:08:03.157840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138344 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:08:03.157840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138348 4615 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:08:03.157840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138353 4615 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:08:03.157840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138357 4615 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:08:03.157840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138362 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:08:03.157840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138366 4615 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:08:03.158620 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138370 4615 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:08:03.158620 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138375 4615 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:08:03.158620 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138379 4615 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:08:03.158620 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138383 4615 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:08:03.158620 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138387 4615 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:08:03.158620 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138392 4615 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:08:03.158620 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138396 4615 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:08:03.158620 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138401 4615 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:08:03.158620 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138405 4615 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:08:03.158620 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138417 4615 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:08:03.158620 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138421 4615 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:08:03.158620 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138425 4615 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:08:03.158620 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138430 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:08:03.158620 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138435 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:08:03.158620 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138439 4615 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:08:03.158620 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138446 4615 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:08:03.158620 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138453 4615 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:08:03.158620 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138458 4615 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:08:03.158620 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138462 4615 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:08:03.159347 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138467 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:08:03.159347 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.138512 4615 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 22:08:03.159347 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138787 4615 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:08:03.159347 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138796 4615 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:08:03.159347 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138801 4615 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:08:03.159347 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138805 4615 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:08:03.159347 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138809 4615 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:08:03.159347 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138813 4615 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:08:03.159347 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138818 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:08:03.159347 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138823 4615 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:08:03.159347 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138828 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:08:03.159347 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138833 4615 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:08:03.159347 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138837 4615 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:08:03.159347 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138842 4615 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:08:03.159347 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138846 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:08:03.159347 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138850 4615 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:08:03.159856 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138854 4615 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:08:03.159856 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138859 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:08:03.159856 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138863 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:08:03.159856 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138870 4615 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:08:03.159856 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138877 4615 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:08:03.159856 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138881 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:08:03.159856 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138886 4615 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:08:03.159856 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138897 4615 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:08:03.159856 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138902 4615 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:08:03.159856 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138908 4615 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:08:03.159856 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138915 4615 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:08:03.159856 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138919 4615 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:08:03.159856 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138924 4615 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:08:03.159856 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138929 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:08:03.159856 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138934 4615 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:08:03.159856 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138938 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:08:03.159856 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138942 4615 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:08:03.159856 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138947 4615 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:08:03.159856 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138952 4615 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:08:03.160351 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138956 4615 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:08:03.160351 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138960 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:08:03.160351 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138965 4615 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:08:03.160351 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138969 4615 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:08:03.160351 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138973 4615 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:08:03.160351 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138978 4615 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:08:03.160351 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138982 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:08:03.160351 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138986 4615 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:08:03.160351 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138991 4615 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:08:03.160351 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138995 4615 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:08:03.160351 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.138998 4615 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:08:03.160351 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139003 4615 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:08:03.160351 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139007 4615 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:08:03.160351 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139011 4615 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:08:03.160351 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139016 4615 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:08:03.160351 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139020 4615 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:08:03.160351 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139024 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:08:03.160351 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139029 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:08:03.160351 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139033 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:08:03.160351 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139037 4615 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:08:03.160834 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139058 4615 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:08:03.160834 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139071 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:08:03.160834 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139076 4615 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:08:03.160834 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139080 4615 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:08:03.160834 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139085 4615 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:08:03.160834 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139089 4615 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:08:03.160834 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139093 4615 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:08:03.160834 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139102 4615 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:08:03.160834 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139107 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:08:03.160834 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139111 4615 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:08:03.160834 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139115 4615 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:08:03.160834 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139120 4615 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:08:03.160834 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139124 4615 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:08:03.160834 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139128 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:08:03.160834 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139133 4615 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:08:03.160834 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139137 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:08:03.160834 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139142 4615 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:08:03.160834 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139146 4615 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:08:03.160834 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139153 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:08:03.161840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139158 4615 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:08:03.161840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139162 4615 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:08:03.161840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139166 4615 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:08:03.161840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139171 4615 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:08:03.161840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139183 4615 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:08:03.161840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139188 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:08:03.161840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139193 4615 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:08:03.161840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139197 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:08:03.161840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139202 4615 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:08:03.161840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139206 4615 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:08:03.161840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139210 4615 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:08:03.161840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139215 4615 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:08:03.161840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139219 4615 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:08:03.161840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139224 4615 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:08:03.161840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139227 4615 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:08:03.161840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139241 4615 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:08:03.161840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139246 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:08:03.161840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139250 4615 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:08:03.161840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139254 4615 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:08:03.161840 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139259 4615 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:08:03.162730 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139265 4615 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:08:03.162730 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139269 4615 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:08:03.162730 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139273 4615 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:08:03.162730 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139278 4615 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:08:03.162730 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139282 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:08:03.162730 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139286 4615 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:08:03.162730 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139290 4615 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:08:03.162730 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139295 4615 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:08:03.162730 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139299 4615 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:08:03.162730 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139303 4615 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:08:03.162730 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139307 4615 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:08:03.162730 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139318 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:08:03.162730 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139322 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:08:03.162730 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139327 4615 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:08:03.162730 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.139371 4615 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 22:08:03.162730 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139431 4615 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 22:08:03.163435 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139439 4615 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 22:08:03.163435 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139444 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 22:08:03.163435 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139449 4615 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 22:08:03.163435 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139453 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 22:08:03.163435 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139458 4615 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 22:08:03.163435 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139462 4615 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 22:08:03.163435 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139466 4615 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 22:08:03.163435 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139471 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 22:08:03.163435 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139475 4615 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 22:08:03.163435 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139479 4615 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 22:08:03.163435 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139484 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 22:08:03.163435 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139488 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 22:08:03.163435 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139506 4615 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 22:08:03.163435 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139514 4615 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 22:08:03.163435 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139519 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 22:08:03.163435 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139524 4615 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 22:08:03.163435 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139530 4615 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 22:08:03.163435 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139536 4615 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 22:08:03.163435 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139543 4615 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 22:08:03.164269 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139548 4615 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 22:08:03.164269 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139553 4615 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 22:08:03.164269 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139558 4615 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 22:08:03.164269 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139563 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 22:08:03.164269 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139567 4615 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 22:08:03.164269 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139572 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 22:08:03.164269 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139576 4615 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 22:08:03.164269 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139580 4615 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 22:08:03.164269 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139587 4615 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 22:08:03.164269 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139591 4615 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 22:08:03.164269 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139596 4615 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 22:08:03.164269 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139601 4615 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 22:08:03.164269 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139605 4615 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 22:08:03.164269 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139609 4615 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 22:08:03.164269 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139614 4615 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 22:08:03.164269 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139618 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 22:08:03.164269 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139622 4615 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 22:08:03.164269 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139630 4615 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 22:08:03.164269 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139641 4615 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 22:08:03.164269 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139646 4615 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 22:08:03.165128 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139650 4615 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 22:08:03.165128 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139654 4615 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 22:08:03.165128 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139659 4615 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 22:08:03.165128 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139663 4615 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 22:08:03.165128 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139668 4615 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 22:08:03.165128 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139672 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 22:08:03.165128 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139684 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 22:08:03.165128 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139689 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 22:08:03.165128 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139693 4615 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 22:08:03.165128 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139697 4615 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 22:08:03.165128 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139703 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 22:08:03.165128 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139708 4615 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 22:08:03.165128 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139712 4615 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 22:08:03.165128 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139716 4615 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 22:08:03.165128 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139720 4615 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 22:08:03.165128 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139724 4615 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 22:08:03.165128 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139729 4615 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 22:08:03.165128 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139733 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 22:08:03.165128 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139737 4615 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 22:08:03.165967 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139741 4615 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 22:08:03.165967 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139748 4615 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 22:08:03.165967 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139752 4615 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 22:08:03.165967 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139756 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 22:08:03.165967 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139761 4615 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 22:08:03.165967 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139765 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 22:08:03.165967 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139770 4615 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 22:08:03.165967 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139774 4615 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 22:08:03.165967 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139779 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 22:08:03.165967 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139783 4615 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 22:08:03.165967 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139788 4615 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 22:08:03.165967 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139792 4615 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 22:08:03.165967 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139797 4615 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 22:08:03.165967 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139801 4615 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 22:08:03.165967 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139806 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 22:08:03.165967 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139810 4615 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 22:08:03.165967 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139814 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 22:08:03.165967 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139818 4615 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 22:08:03.165967 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139823 4615 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 22:08:03.166760 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139827 4615 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 22:08:03.166760 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139839 4615 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 22:08:03.166760 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139844 4615 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 22:08:03.166760 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139848 4615 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 22:08:03.166760 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139854 4615 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 22:08:03.166760 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139858 4615 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 22:08:03.166760 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139862 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 22:08:03.166760 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139867 4615 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 22:08:03.166760 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139871 4615 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 22:08:03.166760 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139876 4615 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 22:08:03.166760 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139880 4615 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 22:08:03.166760 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139884 4615 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 22:08:03.166760 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139889 4615 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 22:08:03.166760 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139893 4615 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 22:08:03.166760 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139899 4615 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 22:08:03.166760 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139903 4615 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 22:08:03.166760 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139907 4615 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 22:08:03.166760 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139912 4615 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 22:08:03.166760 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139916 4615 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 22:08:03.166760 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139921 4615 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 22:08:03.167593 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139925 4615 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 22:08:03.167593 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139929 4615 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 22:08:03.167593 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139933 4615 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 22:08:03.167593 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139938 4615 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 22:08:03.167593 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139942 4615 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 22:08:03.167593 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139948 4615 feature_gate.go:386] unrecognized feature gate: Example Oct 30 22:08:03.167593 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139952 4615 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 22:08:03.167593 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:03.139957 4615 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 22:08:03.167593 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.140067 4615 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 22:08:03.167593 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.141733 4615 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Oct 30 22:08:03.167593 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.141827 4615 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Oct 30 22:08:03.167593 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.142140 4615 server.go:1014] "Starting client certificate rotation" Oct 30 22:08:03.167593 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.142376 4615 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 22:08:03.167593 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.142504 4615 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 22:02:23 +0000 UTC" deadline="2028-04-04 05:19:41.596855502 +0000 UTC" Oct 30 22:08:03.167593 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.142593 4615 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12511h11m38.454266074s" Oct 30 22:08:03.167593 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.143079 4615 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 22:08:03.168349 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.144974 4615 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 22:08:03.168349 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.151730 4615 log.go:25] "Validated CRI v1 runtime API" Oct 30 22:08:03.168349 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.153853 4615 log.go:25] "Validated CRI v1 image API" Oct 30 22:08:03.168349 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.154839 4615 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 22:08:03.168349 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.157030 4615 fs.go:135] Filesystem UUIDs: map[4b5c7890-f881-45d1-ae2c-18e754c8b3dd:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 fef1b876-f40d-4e35-9a1c-db84555c057a:/dev/nvme0n1p4] Oct 30 22:08:03.168565 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.157077 4615 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0172a49ee3a66d8484ba34ef41ebc50fa138bddb747ecdca0564ed33797251fd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0172a49ee3a66d8484ba34ef41ebc50fa138bddb747ecdca0564ed33797251fd/userdata/shm major:0 minor:92 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0a00291d9a933cb4a78a5712b9ece77fd2f46582b2983f5acaba2155f9646c27/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0a00291d9a933cb4a78a5712b9ece77fd2f46582b2983f5acaba2155f9646c27/userdata/shm major:0 minor:211 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/18f67f9fdfc9f0184b71fce97c310c9a17144518445298f23b5e84a23de2dc6e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/18f67f9fdfc9f0184b71fce97c310c9a17144518445298f23b5e84a23de2dc6e/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/458725c86296567a240d35bfb92bb7b3f3b5696cd8f028ef030553c57811e4e1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/458725c86296567a240d35bfb92bb7b3f3b5696cd8f028ef030553c57811e4e1/userdata/shm major:0 minor:219 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/475de44354d8527225668c6098e5464b1da922d27afa961e9923b4865768fb6f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/475de44354d8527225668c6098e5464b1da922d27afa961e9923b4865768fb6f/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4d7c4a4a9633288554efa3923877c1f2118d6d74c526f92b7752344437deefb5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4d7c4a4a9633288554efa3923877c1f2118d6d74c526f92b7752344437deefb5/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/599f89938d5df802596ebac1c4370ccd7550633902cc5876c877b1193910f301/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/599f89938d5df802596ebac1c4370ccd7550633902cc5876c877b1193910f301/userdata/shm major:0 minor:208 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/aa0ca7df9f08416cf33290a4e027b4c09360ae7eead13aebed662755bcdf2607/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/aa0ca7df9f08416cf33290a4e027b4c09360ae7eead13aebed662755bcdf2607/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ab2ecd2f1b5cc900dd679aa3d840dd78546c4e62e7833438c44ce0b6a7ce7731/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ab2ecd2f1b5cc900dd679aa3d840dd78546c4e62e7833438c44ce0b6a7ce7731/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b06c9fab80ab4df18d7da9f617e738d96d9361cf364e83dd6fd16fa5a92923b2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b06c9fab80ab4df18d7da9f617e738d96d9361cf364e83dd6fd16fa5a92923b2/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b394b3d88a7bd12547390ee081799d630407f6779fee7d5c52d926a8fcb48fca/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b394b3d88a7bd12547390ee081799d630407f6779fee7d5c52d926a8fcb48fca/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cea00db705ec975de38c10812f6b5f23be096f68d1eca46642c5f2d97c11ac33/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cea00db705ec975de38c10812f6b5f23be096f68d1eca46642c5f2d97c11ac33/userdata/shm major:0 minor:258 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/dfe98ac968ba7cfb8796c8d56d5f9dc5e8daf9f32802ddaa6ea4d95ebad9cf72/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/dfe98ac968ba7cfb8796c8d56d5f9dc5e8daf9f32802ddaa6ea4d95ebad9cf72/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ea5089406e59eccdcc736a3a1029e09e0c23c74a0931d87a42f891f6fba855f7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ea5089406e59eccdcc736a3a1029e09e0c23c74a0931d87a42f891f6fba855f7/userdata/shm major:0 minor:232 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/eba5686aef29561315892d0735e28949556767cacb184a90b0bc0b66c1182104/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/eba5686aef29561315892d0735e28949556767cacb184a90b0bc0b66c1182104/userdata/shm major:0 minor:233 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f2a1d08519517fcb1d6fe59040ac4cba8e8b764e6ccee10f12cb664f4130f603/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f2a1d08519517fcb1d6fe59040ac4cba8e8b764e6ccee10f12cb664f4130f603/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fe78c30b59feca53bbcfe1307dc8221a5ca0fd965907fc588da59f0e591181ae/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fe78c30b59feca53bbcfe1307dc8221a5ca0fd965907fc588da59f0e591181ae/userdata/shm major:0 minor:77 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/04f40ed9-0eef-4946-b3e6-feafcfee8e03/volumes/kubernetes.io~projected/kube-api-access-szj9w:{mountpoint:/var/lib/kubelet/pods/04f40ed9-0eef-4946-b3e6-feafcfee8e03/volumes/kubernetes.io~projected/kube-api-access-szj9w major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/08904f35-ebda-4220-99c4-b2fb90aa2528/volumes/kubernetes.io~projected/kube-api-access-m2kvz:{mountpoint:/var/lib/kubelet/pods/08904f35-ebda-4220-99c4-b2fb90aa2528/volumes/kubernetes.io~projected/kube-api-access-m2kvz major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/12b2045b-a302-47a1-b039-b8eadcc50c61/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/12b2045b-a302-47a1-b039-b8eadcc50c61/volumes/kubernetes.io~secret/agent-certs major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/24e7e5b3-a4ea-4322-915b-5694ea1f3112/volumes/kubernetes.io~projected/kube-api-access-4pxnh:{mountpoint:/var/lib/kubelet/pods/24e7e5b3-a4ea-4322-915b-5694ea1f3112/volumes/kubernetes.io~projected/kube-api-access-4pxnh major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/24e7e5b3-a4ea-4322-915b-5694ea1f3112/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/24e7e5b3-a4ea-4322-915b-5694ea1f3112/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3bf28a7e-a8ce-4f33-923a-5d1539978790/volumes/kubernetes.io~projected/kube-api-access-6scx6:{mountpoint:/var/lib/kubelet/pods/3bf28a7e-a8ce-4f33-923a-5d1539978790/volumes/kubernetes.io~projected/kube-api-access-6scx6 major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3bf28a7e-a8ce-4f33-923a-5d1539978790/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/3bf28a7e-a8ce-4f33-923a-5d1539978790/volumes/kubernetes.io~secret/metrics-tls major:0 minor:253 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/749dd349-e1d9-4b55-ab5c-5a30739a8cae/volumes/kubernetes.io~projected/kube-api-access-gxhfx:{mountpoint:/var/lib/kubelet/pods/749dd349-e1d9-4b55-ab5c-5a30739a8cae/volumes/kubernetes.io~projected/kube-api-access-gxhfx major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/780de4be-6a4c-4afa-9fee-a3391101d004/volumes/kubernetes.io~projected/kube-api-access-xgtzk:{mountpoint:/var/lib/kubelet/pods/780de4be-6a4c-4afa-9fee-a3391101d004/volumes/kubernetes.io~projected/kube-api-access-xgtzk major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/80423b86-27d7-42d2-9daa-0399bf2303de/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/80423b86-27d7-42d2-9daa-0399bf2303de/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/80423b86-27d7-42d2-9daa-0399bf2303de/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/80423b86-27d7-42d2-9daa-0399bf2303de/volumes/kubernetes.io~empty-dir/tmp major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/80423b86-27d7-42d2-9daa-0399bf2303de/volumes/kubernetes.io~projected/kube-api-access-m2mlc:{mountpoint:/var/lib/kubelet/pods/80423b86-27d7-42d2-9daa-0399bf2303de/volumes/kubernetes.io~projected/kube-api-access-m2mlc major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9432e919-3bed-47a1-8b4e-4b979ebdb046/volumes/kubernetes.io~projected/kube-api-access-m8rz5:{mountpoint:/var/lib/kubelet/pods/9432e919-3bed-47a1-8b4e-4b979ebdb046/volumes/kubernetes.io~projected/kube-api-access-m8rz5 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/95637308-9cdc-4a32-b573-70e1b4cff546/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/95637308-9cdc-4a32-b573-70e1b4cff546/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/95637308-9cdc-4a32-b573-70e1b4cff546/volumes/kubernetes.io~projected/kube-api-access-xt5mm:{mountpoint:/var/lib/kubelet/pods/95637308-9cdc-4a32-b573-70e1b4cff546/volumes/kubernetes.io~projected/kube-api-access-xt5mm major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/95637308-9cdc-4a32-b573-70e1b4cff546/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/95637308-9cdc-4a32-b573-70e1b4cff546/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf/volumes/kubernetes.io~projected/kube-api-access-ffjdk:{mountpoint:/var/lib/kubelet/pods/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf/volumes/kubernetes.io~projected/kube-api-access-ffjdk major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b69c485c-88c0-43c5-a69c-d88c0f81fa22/volumes/kubernetes.io~projected/kube-api-access-q7dg5:{mountpoint:/var/lib/kubelet/pods/b69c485c-88c0-43c5-a69c-d88c0f81fa22/volumes/kubernetes.io~projected/kube-api-access-q7dg5 major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c135a307-61bc-4e34-9b5c-b97793255db7/volumes/kubernetes.io~projected/kube-api-access-8z8f9:{mountpoint:/var/lib/kubelet/pods/c135a307-61bc-4e34-9b5c-b97793255db7/volumes/kubernetes.io~projected/kube-api-access-8z8f9 major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c1f53f6e-aadc-41fd-aa64-20a466fb1769/volumes/kubernetes.io~projected/kube-api-access-kwrm9:{mountpoint:/var/lib/kubelet/pods/c1f53f6e-aadc-41fd-aa64-20a466fb1769/volumes/kubernetes.io~projected/kube-api-access-kwrm9 major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1/volumes/kubernetes.io~projected/kube-api-access-4wrvl:{mountpoint:/var/lib/kubelet/pods/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1/volumes/kubernetes.io~projected/kube-api-access-4wrvl major:0 minor:231 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4/volumes/kubernetes.io~projected/kube-api-access-l28gx:{mountpoint:/var/lib/kubelet/pods/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4/volumes/kubernetes.io~projected/kube-api-access-l28gx major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4/volumes/kubernetes.io~secret/signing-key major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f805f34e-7fc9-430b-9003-16d9765d6657/volumes/kubernetes.io~projected/kube-api-access-scwfv:{mountpoint:/var/lib/kubelet/pods/f805f34e-7fc9-430b-9003-16d9765d6657/volumes/kubernetes.io~projected/kube-api-access-scwfv major:0 minor:57 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/1f1092ad61400eef5aca76303b2a5cac07bbf2738cb232861681ac8a6db8e954/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/c95f486db29bed0cff84ff24939cb2cad041b62d8d1a115bb1f4edda01d6de46/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/427bc3dd6ffc18ea9436e04c9fdc34fb3ec44797b36111888eb315c6b63ff853/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/c5008fb332e0d6c4ddb3ee6483b7fca93599cbdf69762186078ab0c561de9f84/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/2d404b2b7e90c0e503d138953ff2d3beaee3d2646df5ca745dd934af13d2dec2/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/19e42b78b9c920946a6307d09877a159c5125989eddd31a9d5a4c8295b104d8e/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/2f49ee996751c8c1f7fd1edbc5f5cf7d184a32609e94b51db6657367f630d514/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/c059b5de3367a28599ec7ae5be993d2225027e3d846f15e04b736915001a86cb/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/242580688b934c565ec139f486c4cf2ba9009e20642f9b7c2e63e5eabfebefc6/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/d9f743a2cd64e4cd6480ee6247017205ad425a5502ae2727110c62b58a91e857/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-136:{mountpoint:/var/lib/containers/storage/overlay/f7d38b7c8378143726916bec155154f2cb1637b47fa594a6837e922bcde1c6f2/merged major:0 minor:136 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/40e0de1a02c98e33731cc40ac17ff35ef9eea87b80120f62a50b8dea7606c21c/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/ebbbe3b8f8641f3cf907350c1fe5130795190065c038fbc88561453b2c848f51/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/22552ea1044b3aac2c5dbae9f42767d3a3a6e9d8872585323d7a96961c3778a5/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-159:{mountpoint:/var/lib/containers/storage/overlay/72a14022be79a95f37de2550fb882a739ebb3ec418158087a74ec40a56d238a0/merged major:0 minor:159 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/1388e9f630318674135522e58be19f25ef05fcb18f1bd010e08fbd4548d945f8/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/3d7f9a5e41eb70551f4fa5efbfe3e3fffbdba1337f00fed130cdbad5bccc3a83/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/17ab3b851746292fefa74e8809ecc12a62173a06ce057f951981e50d346f3015/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/8a009c9138bb3c5ffdc9afd290ba850e34ddc9ed90f5da6112d6c645d29f84e3/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/d4c2b979c9ed2b8b763647e00a0805c4160668524c041b43c57d5f215f7ba817/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/de2c1b60582f06549cc19928a0ee55e78809627d15233805d3b324b8ebabaea9/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-204:{mountpoint:/var/lib/containers/storage/overlay/6b38fc04c19bb8ccb781a7e749fe0df1d69dcfc1eb12163a56a495f28dd0ed51/merged major:0 minor:204 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/98fcd9cd82d6b47590ada3fe58f646b9ace104cb91a3301c0968ea14e9ab5b1f/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/d808d9ba3f1aabe2dc8586cbf189577ef33836222201fd809ff3b5c5d05b553c/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-221:{mountpoint:/var/lib/containers/storage/overlay/aade13d39c460671d67be8bb452cd15ea737a3c925e6f6f855efcf46fc9598d6/merged major:0 minor:221 fsType:overlay blockSize:0} overlay_0-224:{mountpoint:/var/lib/containers/storage/overlay/63e278e3bb694ab36ce747c88bdc33acd0985aa5970f18787dc179693f4dfc72/merged major:0 minor:224 fsType:overlay blockSize:0} overlay_0-235:{mountpoint:/var/lib/containers/storage/overlay/5e30cc6ccfef58f43837207fff6da226519f99f492a499d78ff6e05b05c216fc/merged major:0 minor:235 fsType:overlay blockSize:0} overlay_0-237:{mountpoint:/var/lib/containers/storage/overlay/cb5f24c855e666f9505c31e15f28e58f5fd14e9354448390c4e7fd15c90aba2d/merged major:0 minor:237 fsType:overlay blockSize:0} overlay_0-239:{mountpoint:/var/lib/containers/storage/overlay/2e5238f345d3c96faf199e21c9625c1bc8906adda774d7e24b2b8844be1324e4/merged major:0 minor:239 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/dcab2022857f4871d97375e1762926d9f7449c1d760396ac78cc73dc89b0b58e/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-247:{mountpoint:/var/lib/containers/storage/overlay/f1b02c31371c3f2d73f4a8302c3c22b77efd5c03b82e1ab4cf50849d6eab3e32/merged major:0 minor:247 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/4c1f3ba66be42a4fdb9fce962808c0f914c38151c28658f794bcd231b72d43e9/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/84b0062dc65214d6950d41cb3720f071dd651ef72ec1becb7b71f1428ec7cf8c/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/cc2ba36feef1c8c7b6e890c6b2708b4eba5344eb98a326ac7175fa9323493560/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-64:{mountpoint:/var/lib/containers/storage/overlay/fc89aafd39410d7acb763d6297f077560a36475bfd64f63d427b49118147a549/merged major:0 minor:64 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/a49f9c020837afca4936b92219efc30bb57829f61638cc82f9fc50c641583d3d/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/23c36a01e39ae6d3a9a21aa0be0f6be20b8114af9d52f4ff191271b4bf985c6a/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/06c80408747f6b3cbf93ab40d3fe8ed38d168661433807c0ffc50eedc31c318a/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/144d9f21c8fc9a01c4ae7a27e0a8730eb9b379e5906fe942d4f3101b8dc1c5fe/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/619e221a11c67328f0ad4656d4ee563a6ea796d84055dc86576228f2e0d1b81c/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/5cb5bd13a19c7bc1eed3e3ef449d1643276a3d9b50130bae2addfffd73ab5b04/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/cfd38b4210691f11d99a62264b32b33889ba3fef707b7ebf24fa5d9341271c9d/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/e0c14cd84c3868e33285bf4a117de73a195461605ebf9da622d082c29be8080e/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/03dfb1cc1a1fa73acd8081b47e7113ef584c913e39551cd8c38e5bebffae154d/merged major:0 minor:96 fsType:overlay blockSize:0}] Oct 30 22:08:03.168805 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.166710 4615 manager.go:217] Machine: {Timestamp:2026-10-30 22:08:03.165988323 +0000 UTC m=+0.081674620 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100210 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bafde1fd402ce4526e077dca30f75 SystemUUID:ec2bafde-1fd4-02ce-4526-e077dca30f75 BootID:458000f1-c64c-476b-b69d-b4ec70cc87b2 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f805f34e-7fc9-430b-9003-16d9765d6657/volumes/kubernetes.io~projected/kube-api-access-scwfv DeviceMajor:0 DeviceMinor:57 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/475de44354d8527225668c6098e5464b1da922d27afa961e9923b4865768fb6f/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/95637308-9cdc-4a32-b573-70e1b4cff546/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/458725c86296567a240d35bfb92bb7b3f3b5696cd8f028ef030553c57811e4e1/userdata/shm DeviceMajor:0 DeviceMinor:219 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1/volumes/kubernetes.io~projected/kube-api-access-4wrvl DeviceMajor:0 DeviceMinor:231 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-237 DeviceMajor:0 DeviceMinor:237 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-64 DeviceMajor:0 DeviceMinor:64 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f2a1d08519517fcb1d6fe59040ac4cba8e8b764e6ccee10f12cb664f4130f603/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0a00291d9a933cb4a78a5712b9ece77fd2f46582b2983f5acaba2155f9646c27/userdata/shm DeviceMajor:0 DeviceMinor:211 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4/volumes/kubernetes.io~projected/kube-api-access-l28gx DeviceMajor:0 DeviceMinor:216 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:223 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/dfe98ac968ba7cfb8796c8d56d5f9dc5e8daf9f32802ddaa6ea4d95ebad9cf72/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/80423b86-27d7-42d2-9daa-0399bf2303de/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:50 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/18f67f9fdfc9f0184b71fce97c310c9a17144518445298f23b5e84a23de2dc6e/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf/volumes/kubernetes.io~projected/kube-api-access-ffjdk DeviceMajor:0 DeviceMinor:54 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/24e7e5b3-a4ea-4322-915b-5694ea1f3112/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/80423b86-27d7-42d2-9daa-0399bf2303de/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:48 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0172a49ee3a66d8484ba34ef41ebc50fa138bddb747ecdca0564ed33797251fd/userdata/shm DeviceMajor:0 DeviceMinor:92 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/aa0ca7df9f08416cf33290a4e027b4c09360ae7eead13aebed662755bcdf2607/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/780de4be-6a4c-4afa-9fee-a3391101d004/volumes/kubernetes.io~projected/kube-api-access-xgtzk DeviceMajor:0 DeviceMinor:206 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/12b2045b-a302-47a1-b039-b8eadcc50c61/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:47 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/b69c485c-88c0-43c5-a69c-d88c0f81fa22/volumes/kubernetes.io~projected/kube-api-access-q7dg5 DeviceMajor:0 DeviceMinor:55 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/95637308-9cdc-4a32-b573-70e1b4cff546/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:49 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/749dd349-e1d9-4b55-ab5c-5a30739a8cae/volumes/kubernetes.io~projected/kube-api-access-gxhfx DeviceMajor:0 DeviceMinor:56 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4d7c4a4a9633288554efa3923877c1f2118d6d74c526f92b7752344437deefb5/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fe78c30b59feca53bbcfe1307dc8221a5ca0fd965907fc588da59f0e591181ae/userdata/shm DeviceMajor:0 DeviceMinor:77 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3bf28a7e-a8ce-4f33-923a-5d1539978790/volumes/kubernetes.io~projected/kube-api-access-6scx6 DeviceMajor:0 DeviceMinor:201 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:215 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-136 DeviceMajor:0 DeviceMinor:136 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/80423b86-27d7-42d2-9daa-0399bf2303de/volumes/kubernetes.io~projected/kube-api-access-m2mlc DeviceMajor:0 DeviceMinor:58 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/9432e919-3bed-47a1-8b4e-4b979ebdb046/volumes/kubernetes.io~projected/kube-api-access-m8rz5 DeviceMajor:0 DeviceMinor:59 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/95637308-9cdc-4a32-b573-70e1b4cff546/volumes/kubernetes.io~projected/kube-api-access-xt5mm DeviceMajor:0 DeviceMinor:51 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/24e7e5b3-a4ea-4322-915b-5694ea1f3112/volumes/kubernetes.io~projected/kube-api-access-4pxnh DeviceMajor:0 DeviceMinor:197 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-204 DeviceMajor:0 DeviceMinor:204 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-221 DeviceMajor:0 DeviceMinor:221 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-224 DeviceMajor:0 DeviceMinor:224 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-235 DeviceMajor:0 DeviceMinor:235 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b06c9fab80ab4df18d7da9f617e738d96d9361cf364e83dd6fd16fa5a92923b2/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c1f53f6e-aadc-41fd-aa64-20a466fb1769/volumes/kubernetes.io~projected/kube-api-access-kwrm9 DeviceMajor:0 DeviceMinor:210 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3bf28a7e-a8ce-4f33-923a-5d1539978790/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:253 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/08904f35-ebda-4220-99c4-b2fb90aa2528/volumes/kubernetes.io~projected/kube-api-access-m2kvz DeviceMajor:0 DeviceMinor:53 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ab2ecd2f1b5cc900dd679aa3d840dd78546c4e62e7833438c44ce0b6a7ce7731/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/04f40ed9-0eef-4946-b3e6-feafcfee8e03/volumes/kubernetes.io~projected/kube-api-access-szj9w DeviceMajor:0 DeviceMinor:207 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/599f89938d5df802596ebac1c4370ccd7550633902cc5876c877b1193910f301/userdata/shm DeviceMajor:0 DeviceMinor:208 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/c135a307-61bc-4e34-9b5c-b97793255db7/volumes/kubernetes.io~projected/kube-api-access-8z8f9 DeviceMajor:0 DeviceMinor:52 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-159 DeviceMajor:0 DeviceMinor:159 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-239 DeviceMajor:0 DeviceMinor:239 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-247 DeviceMajor:0 DeviceMinor:247 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cea00db705ec975de38c10812f6b5f23be096f68d1eca46642c5f2d97c11ac33/userdata/shm DeviceMajor:0 DeviceMinor:258 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b394b3d88a7bd12547390ee081799d630407f6779fee7d5c52d926a8fcb48fca/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ea5089406e59eccdcc736a3a1029e09e0c23c74a0931d87a42f891f6fba855f7/userdata/shm DeviceMajor:0 DeviceMinor:232 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/eba5686aef29561315892d0735e28949556767cacb184a90b0bc0b66c1182104/userdata/shm DeviceMajor:0 DeviceMinor:233 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:0a00291d9a933cb MacAddress:b2:8d:4e:6a:2f:3b Speed:10000 Mtu:8901} {Name:458725c86296567 MacAddress:16:5b:1a:f3:17:6e Speed:10000 Mtu:8901} {Name:599f89938d5df80 MacAddress:4a:d0:df:cb:74:89 Speed:10000 Mtu:8901} {Name:aa0ca7df9f08416 MacAddress:72:fe:e3:e7:d4:e3 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:db:d1:4a:ef Speed:0 Mtu:9001} {Name:br-int MacAddress:96:f3:a3:1f:9f:4c Speed:0 Mtu:8901} {Name:cea00db705ec975 MacAddress:c6:51:d6:0c:d5:e1 Speed:10000 Mtu:8901} {Name:ea5089406e59ecc MacAddress:22:b6:d0:b1:58:85 Speed:10000 Mtu:8901} {Name:eba5686aef29561 MacAddress:46:35:a5:9e:af:95 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:db:d1:4a:ef Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:7e:82:c1:9c:b2:25 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:62:92:fb:dd:1e:95 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 22:08:03.168805 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.167158 4615 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 22:08:03.168805 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.167222 4615 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 22:08:03.168805 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.167664 4615 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 22:08:03.168805 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.167700 4615 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-94.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 22:08:03.168805 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.167919 4615 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 22:08:03.168805 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.167932 4615 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 22:08:03.168805 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.167947 4615 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 22:08:03.168805 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.167970 4615 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 22:08:03.168805 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.168124 4615 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 22:08:03.168805 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.168437 4615 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 22:08:03.168805 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.168571 4615 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 22:08:03.168805 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.168610 4615 kubelet.go:493] "Attempting to sync node with API server" Oct 30 22:08:03.168805 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.168625 4615 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 22:08:03.168805 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.168640 4615 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 22:08:03.168805 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.168654 4615 kubelet.go:405] "Adding apiserver pod source" Oct 30 22:08:03.168805 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.168666 4615 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 22:08:03.169626 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.169448 4615 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 22:08:03.169626 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.169463 4615 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 22:08:03.173480 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.173459 4615 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 22:08:03.187151 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.187121 4615 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 22:08:03.187151 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.187148 4615 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 22:08:03.187291 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.187258 4615 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Oct 30 22:08:03.187458 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.187446 4615 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 22:08:03.187487 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.187477 4615 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 22:08:03.187584 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.187575 4615 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 22:08:03.187678 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.187591 4615 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 22:08:03.187678 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.187597 4615 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 22:08:03.187678 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.187604 4615 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 22:08:03.187678 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.187610 4615 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 22:08:03.187678 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.187616 4615 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 22:08:03.187678 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.187626 4615 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 22:08:03.187678 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.187640 4615 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 22:08:03.187678 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.187649 4615 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 22:08:03.187678 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.187655 4615 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 22:08:03.187678 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.187674 4615 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 22:08:03.187678 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.187683 4615 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 22:08:03.187979 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.187711 4615 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 22:08:03.187979 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.187719 4615 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 22:08:03.191477 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.191456 4615 server.go:1263] "Started kubelet" Oct 30 22:08:03.191589 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.191513 4615 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 22:08:03.195831 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.192014 4615 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 22:08:03.195831 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.192124 4615 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 22:08:03.195831 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.194876 4615 server.go:317] "Adding debug handlers to kubelet server" Oct 30 22:08:03.195831 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.195364 4615 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 22:08:03.192269 ip-10-0-131-94 systemd[1]: Started Kubernetes Kubelet. Oct 30 22:08:03.196693 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.196672 4615 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 22:08:03.196778 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.196715 4615 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 22:08:03.196778 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.196761 4615 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 22:02:24 +0000 UTC" deadline="2028-08-07 06:44:42.30535572 +0000 UTC" Oct 30 22:08:03.196869 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.196779 4615 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15512h36m39.108579194s" Oct 30 22:08:03.196930 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.196870 4615 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 22:08:03.196930 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.196878 4615 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 22:08:03.196930 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:08:03.196898 4615 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-94.ec2.internal\" not found" Oct 30 22:08:03.198931 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.198910 4615 factory.go:55] Registering systemd factory Oct 30 22:08:03.198931 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.198932 4615 factory.go:223] Registration of the systemd container factory successfully Oct 30 22:08:03.199367 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.199351 4615 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 22:08:03.200508 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.199520 4615 factory.go:153] Registering CRI-O factory Oct 30 22:08:03.200508 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.199538 4615 factory.go:223] Registration of the crio container factory successfully Oct 30 22:08:03.200508 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.199590 4615 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 22:08:03.200508 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.199608 4615 factory.go:103] Registering Raw factory Oct 30 22:08:03.200508 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.199625 4615 manager.go:1199] Started watching for new ooms in manager Oct 30 22:08:03.200508 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.199972 4615 manager.go:319] Starting recovery of all containers Oct 30 22:08:03.201324 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201283 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="04f40ed9-0eef-4946-b3e6-feafcfee8e03" volumeName="kubernetes.io/configmap/04f40ed9-0eef-4946-b3e6-feafcfee8e03-service-ca-bundle" seLinuxMountContext="" Oct 30 22:08:03.201423 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201328 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="12b2045b-a302-47a1-b039-b8eadcc50c61" volumeName="kubernetes.io/secret/12b2045b-a302-47a1-b039-b8eadcc50c61-agent-certs" seLinuxMountContext="" Oct 30 22:08:03.201423 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201345 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="24e7e5b3-a4ea-4322-915b-5694ea1f3112" volumeName="kubernetes.io/secret/24e7e5b3-a4ea-4322-915b-5694ea1f3112-cert" seLinuxMountContext="" Oct 30 22:08:03.201423 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201359 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3bf28a7e-a8ce-4f33-923a-5d1539978790" volumeName="kubernetes.io/configmap/3bf28a7e-a8ce-4f33-923a-5d1539978790-config-volume" seLinuxMountContext="" Oct 30 22:08:03.201423 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201374 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3bf28a7e-a8ce-4f33-923a-5d1539978790" volumeName="kubernetes.io/projected/3bf28a7e-a8ce-4f33-923a-5d1539978790-kube-api-access-6scx6" seLinuxMountContext="" Oct 30 22:08:03.201423 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201390 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="749dd349-e1d9-4b55-ab5c-5a30739a8cae" volumeName="kubernetes.io/projected/749dd349-e1d9-4b55-ab5c-5a30739a8cae-kube-api-access-gxhfx" seLinuxMountContext="" Oct 30 22:08:03.201423 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201402 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b69c485c-88c0-43c5-a69c-d88c0f81fa22" volumeName="kubernetes.io/configmap/b69c485c-88c0-43c5-a69c-d88c0f81fa22-whereabouts-flatfile-configmap" seLinuxMountContext="" Oct 30 22:08:03.201423 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201417 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="04f40ed9-0eef-4946-b3e6-feafcfee8e03" volumeName="kubernetes.io/configmap/04f40ed9-0eef-4946-b3e6-feafcfee8e03-trusted-ca-bundle" seLinuxMountContext="" Oct 30 22:08:03.201757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201434 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="780de4be-6a4c-4afa-9fee-a3391101d004" volumeName="kubernetes.io/configmap/780de4be-6a4c-4afa-9fee-a3391101d004-kube-rbac-proxy-cm" seLinuxMountContext="" Oct 30 22:08:03.201757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201447 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="95637308-9cdc-4a32-b573-70e1b4cff546" volumeName="kubernetes.io/configmap/95637308-9cdc-4a32-b573-70e1b4cff546-env-overrides" seLinuxMountContext="" Oct 30 22:08:03.201757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201462 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="08904f35-ebda-4220-99c4-b2fb90aa2528" volumeName="kubernetes.io/configmap/08904f35-ebda-4220-99c4-b2fb90aa2528-iptables-alerter-script" seLinuxMountContext="" Oct 30 22:08:03.201757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201474 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="780de4be-6a4c-4afa-9fee-a3391101d004" volumeName="kubernetes.io/empty-dir/780de4be-6a4c-4afa-9fee-a3391101d004-data-volume" seLinuxMountContext="" Oct 30 22:08:03.201757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201493 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="95637308-9cdc-4a32-b573-70e1b4cff546" volumeName="kubernetes.io/secret/95637308-9cdc-4a32-b573-70e1b4cff546-ovn-node-metrics-cert" seLinuxMountContext="" Oct 30 22:08:03.201757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201531 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9de98c42-a20f-4d6f-b8ab-d0cabf70dedf" volumeName="kubernetes.io/configmap/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-daemon-config" seLinuxMountContext="" Oct 30 22:08:03.201757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201550 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b69c485c-88c0-43c5-a69c-d88c0f81fa22" volumeName="kubernetes.io/configmap/b69c485c-88c0-43c5-a69c-d88c0f81fa22-cni-sysctl-allowlist" seLinuxMountContext="" Oct 30 22:08:03.201757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201562 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9" volumeName="kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret" seLinuxMountContext="" Oct 30 22:08:03.201757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201575 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c135a307-61bc-4e34-9b5c-b97793255db7" volumeName="kubernetes.io/projected/c135a307-61bc-4e34-9b5c-b97793255db7-kube-api-access-8z8f9" seLinuxMountContext="" Oct 30 22:08:03.201757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201587 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4" volumeName="kubernetes.io/projected/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4-kube-api-access-l28gx" seLinuxMountContext="" Oct 30 22:08:03.201757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201600 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="08904f35-ebda-4220-99c4-b2fb90aa2528" volumeName="kubernetes.io/projected/08904f35-ebda-4220-99c4-b2fb90aa2528-kube-api-access-m2kvz" seLinuxMountContext="" Oct 30 22:08:03.201757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201613 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="80423b86-27d7-42d2-9daa-0399bf2303de" volumeName="kubernetes.io/projected/80423b86-27d7-42d2-9daa-0399bf2303de-kube-api-access-m2mlc" seLinuxMountContext="" Oct 30 22:08:03.201757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201626 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b69c485c-88c0-43c5-a69c-d88c0f81fa22" volumeName="kubernetes.io/configmap/b69c485c-88c0-43c5-a69c-d88c0f81fa22-cni-binary-copy" seLinuxMountContext="" Oct 30 22:08:03.201757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201638 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4" volumeName="kubernetes.io/secret/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4-signing-key" seLinuxMountContext="" Oct 30 22:08:03.201757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201651 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9432e919-3bed-47a1-8b4e-4b979ebdb046" volumeName="kubernetes.io/projected/9432e919-3bed-47a1-8b4e-4b979ebdb046-kube-api-access-m8rz5" seLinuxMountContext="" Oct 30 22:08:03.201757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201664 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="95637308-9cdc-4a32-b573-70e1b4cff546" volumeName="kubernetes.io/configmap/95637308-9cdc-4a32-b573-70e1b4cff546-ovnkube-config" seLinuxMountContext="" Oct 30 22:08:03.201757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201678 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="95637308-9cdc-4a32-b573-70e1b4cff546" volumeName="kubernetes.io/projected/95637308-9cdc-4a32-b573-70e1b4cff546-kube-api-access-xt5mm" seLinuxMountContext="" Oct 30 22:08:03.201757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201695 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="80423b86-27d7-42d2-9daa-0399bf2303de" volumeName="kubernetes.io/empty-dir/80423b86-27d7-42d2-9daa-0399bf2303de-etc-tuned" seLinuxMountContext="" Oct 30 22:08:03.201757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201708 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9432e919-3bed-47a1-8b4e-4b979ebdb046" volumeName="kubernetes.io/configmap/9432e919-3bed-47a1-8b4e-4b979ebdb046-serviceca" seLinuxMountContext="" Oct 30 22:08:03.201757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201721 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b69c485c-88c0-43c5-a69c-d88c0f81fa22" volumeName="kubernetes.io/projected/b69c485c-88c0-43c5-a69c-d88c0f81fa22-kube-api-access-q7dg5" seLinuxMountContext="" Oct 30 22:08:03.201757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201738 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1f53f6e-aadc-41fd-aa64-20a466fb1769" volumeName="kubernetes.io/projected/c1f53f6e-aadc-41fd-aa64-20a466fb1769-kube-api-access-kwrm9" seLinuxMountContext="" Oct 30 22:08:03.201757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201759 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="24e7e5b3-a4ea-4322-915b-5694ea1f3112" volumeName="kubernetes.io/projected/24e7e5b3-a4ea-4322-915b-5694ea1f3112-kube-api-access-4pxnh" seLinuxMountContext="" Oct 30 22:08:03.202650 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201785 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3bf28a7e-a8ce-4f33-923a-5d1539978790" volumeName="kubernetes.io/secret/3bf28a7e-a8ce-4f33-923a-5d1539978790-metrics-tls" seLinuxMountContext="" Oct 30 22:08:03.202650 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201810 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="780de4be-6a4c-4afa-9fee-a3391101d004" volumeName="kubernetes.io/projected/780de4be-6a4c-4afa-9fee-a3391101d004-kube-api-access-xgtzk" seLinuxMountContext="" Oct 30 22:08:03.202650 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201823 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="80423b86-27d7-42d2-9daa-0399bf2303de" volumeName="kubernetes.io/empty-dir/80423b86-27d7-42d2-9daa-0399bf2303de-tmp" seLinuxMountContext="" Oct 30 22:08:03.202650 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201837 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="04f40ed9-0eef-4946-b3e6-feafcfee8e03" volumeName="kubernetes.io/empty-dir/04f40ed9-0eef-4946-b3e6-feafcfee8e03-archives-path" seLinuxMountContext="" Oct 30 22:08:03.202650 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201861 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9de98c42-a20f-4d6f-b8ab-d0cabf70dedf" volumeName="kubernetes.io/projected/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-kube-api-access-ffjdk" seLinuxMountContext="" Oct 30 22:08:03.202650 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201900 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c135a307-61bc-4e34-9b5c-b97793255db7" volumeName="kubernetes.io/empty-dir/c135a307-61bc-4e34-9b5c-b97793255db7-tmp-dir" seLinuxMountContext="" Oct 30 22:08:03.202650 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201914 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d3d3d52b-f209-4ad4-b531-4fe19bad4bf1" volumeName="kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl" seLinuxMountContext="" Oct 30 22:08:03.202650 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201928 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4" volumeName="kubernetes.io/configmap/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4-signing-cabundle" seLinuxMountContext="" Oct 30 22:08:03.202650 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201941 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="04f40ed9-0eef-4946-b3e6-feafcfee8e03" volumeName="kubernetes.io/projected/04f40ed9-0eef-4946-b3e6-feafcfee8e03-kube-api-access-szj9w" seLinuxMountContext="" Oct 30 22:08:03.202650 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201954 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="12b2045b-a302-47a1-b039-b8eadcc50c61" volumeName="kubernetes.io/configmap/12b2045b-a302-47a1-b039-b8eadcc50c61-konnectivity-ca" seLinuxMountContext="" Oct 30 22:08:03.202650 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201968 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3bf28a7e-a8ce-4f33-923a-5d1539978790" volumeName="kubernetes.io/empty-dir/3bf28a7e-a8ce-4f33-923a-5d1539978790-tmp-dir" seLinuxMountContext="" Oct 30 22:08:03.202650 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201980 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="95637308-9cdc-4a32-b573-70e1b4cff546" volumeName="kubernetes.io/configmap/95637308-9cdc-4a32-b573-70e1b4cff546-ovnkube-script-lib" seLinuxMountContext="" Oct 30 22:08:03.202650 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.201995 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9de98c42-a20f-4d6f-b8ab-d0cabf70dedf" volumeName="kubernetes.io/configmap/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-cni-binary-copy" seLinuxMountContext="" Oct 30 22:08:03.202650 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.202008 4615 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f805f34e-7fc9-430b-9003-16d9765d6657" volumeName="kubernetes.io/projected/f805f34e-7fc9-430b-9003-16d9765d6657-kube-api-access-scwfv" seLinuxMountContext="" Oct 30 22:08:03.202650 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.202021 4615 reconstruct.go:97] "Volume reconstruction finished" Oct 30 22:08:03.202650 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.202029 4615 reconciler.go:29] "Reconciler: start to sync state" Oct 30 22:08:03.217603 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.217573 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:08:03.223961 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.223926 4615 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 22:08:03.226235 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.226196 4615 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 22:08:03.227499 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.227475 4615 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 22:08:03.227499 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.227496 4615 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 22:08:03.227688 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.227522 4615 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 22:08:03.227688 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:08:03.227565 4615 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Oct 30 22:08:03.232594 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.232572 4615 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 22:08:03.234880 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.234853 4615 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 22:08:03.235941 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.235852 4615 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Oct 30 22:08:03.244029 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.243958 4615 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 22:08:03.244448 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.244406 4615 generic.go:356] "Generic (PLEG): container finished" podID="00a20884b67d7fdeb7bbc7e471370a8a" containerID="297f568d42424df7a502b35ac0d6b8dd3dadc151f32d4205f6a783c16e5d4013" exitCode=0 Oct 30 22:08:03.247915 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.247896 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:08:03.255709 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.255679 4615 generic.go:356] "Generic (PLEG): container finished" podID="b69c485c-88c0-43c5-a69c-d88c0f81fa22" containerID="bce12887bfc462cf9745baee6f60759ab78522159f86d263eb3cbd65704b3a76" exitCode=0 Oct 30 22:08:03.255709 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.255710 4615 generic.go:356] "Generic (PLEG): container finished" podID="b69c485c-88c0-43c5-a69c-d88c0f81fa22" containerID="d6d18042c05990ff20ed562345e676c46c8cf70448d3751e31f2624c129f9c8f" exitCode=0 Oct 30 22:08:03.255832 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.255723 4615 generic.go:356] "Generic (PLEG): container finished" podID="b69c485c-88c0-43c5-a69c-d88c0f81fa22" containerID="a9d3d2a7cb4c13878d20d79b20c82781a4d7fea6517cc581c24f8ad94c0457b3" exitCode=0 Oct 30 22:08:03.255832 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.255732 4615 generic.go:356] "Generic (PLEG): container finished" podID="b69c485c-88c0-43c5-a69c-d88c0f81fa22" containerID="177c2909abaca0af49fd5adeec9fc16e609b38bd8a95136dca8772dbb8172abb" exitCode=0 Oct 30 22:08:03.255832 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.255745 4615 generic.go:356] "Generic (PLEG): container finished" podID="b69c485c-88c0-43c5-a69c-d88c0f81fa22" containerID="53542bbdbecb2d73c6ed64bc560df3290657396e85bdf49443db1c30453f27de" exitCode=0 Oct 30 22:08:03.255832 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.255753 4615 generic.go:356] "Generic (PLEG): container finished" podID="b69c485c-88c0-43c5-a69c-d88c0f81fa22" containerID="a303caee0480d05cccf6109f690fa1715f1337d7954b61e643478f70dbd51148" exitCode=0 Oct 30 22:08:03.264563 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.264538 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:08:03.264890 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.264866 4615 generic.go:356] "Generic (PLEG): container finished" podID="95637308-9cdc-4a32-b573-70e1b4cff546" containerID="fd271464d9e5a6c0303c47684099968b3bf6a61bf4fc98388a40041007d71b3a" exitCode=1 Oct 30 22:08:03.277020 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.276997 4615 manager.go:324] Recovery completed Oct 30 22:08:03.293454 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.293425 4615 cpu_manager.go:226] "Starting" policy="none" Oct 30 22:08:03.293454 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.293443 4615 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 22:08:03.293675 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.293465 4615 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 22:08:03.294114 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.293635 4615 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Oct 30 22:08:03.294214 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.294115 4615 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Oct 30 22:08:03.294214 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.294145 4615 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Oct 30 22:08:03.294214 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.294154 4615 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Oct 30 22:08:03.294214 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.294162 4615 policy_none.go:50] "Start" Oct 30 22:08:03.294214 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.294181 4615 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 22:08:03.294214 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.294196 4615 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 22:08:03.294492 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.294385 4615 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Oct 30 22:08:03.294492 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.294396 4615 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Oct 30 22:08:03.294492 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.294404 4615 policy_none.go:44] "Start" Oct 30 22:08:03.298443 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.298414 4615 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 22:08:03.302107 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.302093 4615 manager.go:341] "Starting Device Plugin manager" Oct 30 22:08:03.302231 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:08:03.302120 4615 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 22:08:03.302231 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.302128 4615 server.go:87] "Starting device plugin registration server" Oct 30 22:08:03.302416 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.302405 4615 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 22:08:03.302451 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.302419 4615 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 22:08:03.302526 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.302509 4615 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 22:08:03.302644 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.302607 4615 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 22:08:03.302685 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.302661 4615 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 22:08:03.302685 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.302668 4615 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 22:08:03.302786 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.302717 4615 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T22:08:03.302642564Z","UUID":"4b7e0811-e1f8-4cb3-a7d6-9e8c93d774c2","Handler":null,"Name":"","Endpoint":""} Oct 30 22:08:03.303995 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.303978 4615 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 22:08:03.304098 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.304004 4615 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 22:08:03.304400 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:08:03.304372 4615 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 22:08:03.327701 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.327651 4615 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal"] Oct 30 22:08:03.328153 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.328123 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" event={"ID":"f39ce36b79a5ed24b44c91f0e6e003f7","Type":"ContainerStarted","Data":"3bb33b8d00383e9f2982a9dde4b652ba8dd5ec9ffe8cd9ea54e5c41f0f694f1f"} Oct 30 22:08:03.328211 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.328160 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" event={"ID":"f39ce36b79a5ed24b44c91f0e6e003f7","Type":"ContainerStarted","Data":"b06c9fab80ab4df18d7da9f617e738d96d9361cf364e83dd6fd16fa5a92923b2"} Oct 30 22:08:03.328211 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.328173 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" event={"ID":"00a20884b67d7fdeb7bbc7e471370a8a","Type":"ContainerStarted","Data":"fa5bf0103648429155c90d187a318e250d1f61b09d5bbc330afeea5e20e37984"} Oct 30 22:08:03.328211 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.328184 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" event={"ID":"00a20884b67d7fdeb7bbc7e471370a8a","Type":"ContainerDied","Data":"297f568d42424df7a502b35ac0d6b8dd3dadc151f32d4205f6a783c16e5d4013"} Oct 30 22:08:03.328211 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.328199 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" event={"ID":"00a20884b67d7fdeb7bbc7e471370a8a","Type":"ContainerStarted","Data":"475de44354d8527225668c6098e5464b1da922d27afa961e9923b4865768fb6f"} Oct 30 22:08:03.328331 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.328242 4615 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" Oct 30 22:08:03.328389 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.328374 4615 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Oct 30 22:08:03.347019 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.346990 4615 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 22:08:03.347168 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:08:03.347061 4615 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-131-94.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" Oct 30 22:08:03.347586 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.347568 4615 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 22:08:03.347618 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:08:03.347607 4615 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Oct 30 22:08:03.403430 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.403396 4615 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 22:08:03.404790 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.404767 4615 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientMemory" Oct 30 22:08:03.404901 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.404805 4615 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 22:08:03.404901 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.404819 4615 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-94.ec2.internal" event="NodeHasSufficientPID" Oct 30 22:08:03.404901 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.404853 4615 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-131-94.ec2.internal" Oct 30 22:08:03.435487 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.435453 4615 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-131-94.ec2.internal" Oct 30 22:08:03.435652 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.435538 4615 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-131-94.ec2.internal" Oct 30 22:08:03.437199 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.437172 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f39ce36b79a5ed24b44c91f0e6e003f7-config\") pod \"kube-apiserver-proxy-ip-10-0-131-94.ec2.internal\" (UID: \"f39ce36b79a5ed24b44c91f0e6e003f7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" Oct 30 22:08:03.437298 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.437207 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/00a20884b67d7fdeb7bbc7e471370a8a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal\" (UID: \"00a20884b67d7fdeb7bbc7e471370a8a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Oct 30 22:08:03.437298 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.437259 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00a20884b67d7fdeb7bbc7e471370a8a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal\" (UID: \"00a20884b67d7fdeb7bbc7e471370a8a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Oct 30 22:08:03.537918 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.537841 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/00a20884b67d7fdeb7bbc7e471370a8a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal\" (UID: \"00a20884b67d7fdeb7bbc7e471370a8a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Oct 30 22:08:03.537918 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.537891 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00a20884b67d7fdeb7bbc7e471370a8a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal\" (UID: \"00a20884b67d7fdeb7bbc7e471370a8a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Oct 30 22:08:03.537918 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.537919 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f39ce36b79a5ed24b44c91f0e6e003f7-config\") pod \"kube-apiserver-proxy-ip-10-0-131-94.ec2.internal\" (UID: \"f39ce36b79a5ed24b44c91f0e6e003f7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" Oct 30 22:08:03.538151 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.537851 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/00a20884b67d7fdeb7bbc7e471370a8a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal\" (UID: \"00a20884b67d7fdeb7bbc7e471370a8a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Oct 30 22:08:03.538151 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.537966 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f39ce36b79a5ed24b44c91f0e6e003f7-config\") pod \"kube-apiserver-proxy-ip-10-0-131-94.ec2.internal\" (UID: \"f39ce36b79a5ed24b44c91f0e6e003f7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" Oct 30 22:08:03.538151 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:03.537983 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00a20884b67d7fdeb7bbc7e471370a8a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal\" (UID: \"00a20884b67d7fdeb7bbc7e471370a8a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Oct 30 22:08:04.174239 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.174188 4615 apiserver.go:52] "Watching apiserver" Oct 30 22:08:04.210150 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.210120 4615 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 22:08:04.210564 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.210545 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-gsvm9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal","openshift-kube-storage-version-migrator/migrator-66df5d496b-lc6lf","kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal","openshift-image-registry/node-ca-h9qqf","openshift-ingress-canary/ingress-canary-khpnv","openshift-dns/dns-default-xtbrk","openshift-multus/multus-4gdsr","openshift-dns/node-resolver-q5gw7","openshift-multus/network-metrics-daemon-l7k26","openshift-multus/multus-additional-cni-plugins-lzh8c","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79","openshift-insights/insights-runtime-extractor-gs6ts","openshift-service-ca/service-ca-5fffc97bdf-69qgk","openshift-network-diagnostics/network-check-target-pmkb2","openshift-ovn-kubernetes/ovnkube-node-wqj4x","kube-system/konnectivity-agent-6qz5h","openshift-cluster-node-tuning-operator/tuned-w692c","openshift-insights/periodic-gathering-r9szz-8wv8c","openshift-network-operator/iptables-alerter-zt9w6"] Oct 30 22:08:04.213441 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.213422 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:08:04.219316 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.219294 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:08:04.224657 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.224633 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 22:08:04.224657 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.224647 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 22:08:04.224771 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.224656 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 22:08:04.224771 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.224647 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 22:08:04.224771 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.224709 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4vl25\"" Oct 30 22:08:04.224867 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.224784 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-xh2wl\"" Oct 30 22:08:04.224904 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.224868 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 22:08:04.224972 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.224960 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 22:08:04.233443 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.233418 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 22:08:04.233632 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.233469 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 22:08:04.233632 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.233480 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zxnxl\"" Oct 30 22:08:04.233632 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.233515 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 22:08:04.233632 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.233521 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-g99sl\"" Oct 30 22:08:04.233632 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.233531 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 22:08:04.233632 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.233579 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 22:08:04.233632 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.233596 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 22:08:04.233881 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.233749 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 22:08:04.233881 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.233848 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 22:08:04.233881 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.233851 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 22:08:04.233994 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.233899 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 22:08:04.233994 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.233940 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 22:08:04.234069 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.234005 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-vngtk\"" Oct 30 22:08:04.234439 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.234426 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 22:08:04.234485 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.234465 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 22:08:04.234624 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.234612 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 22:08:04.234939 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.234922 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Oct 30 22:08:04.235019 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.234948 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 22:08:04.235078 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235024 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5n25k\"" Oct 30 22:08:04.235111 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235092 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 22:08:04.235246 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235232 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Oct 30 22:08:04.235302 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235266 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 22:08:04.235302 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235287 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-l4sc9\"" Oct 30 22:08:04.235452 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235432 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 22:08:04.235452 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235446 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 22:08:04.235543 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235457 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 22:08:04.235543 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235435 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 22:08:04.235543 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235501 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 22:08:04.235543 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235508 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 22:08:04.235543 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235517 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 22:08:04.235768 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235545 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Oct 30 22:08:04.235768 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235682 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 22:08:04.235768 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235732 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 22:08:04.235768 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235748 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-zf4xw\"" Oct 30 22:08:04.235768 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235751 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 22:08:04.235927 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235773 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 22:08:04.235927 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235799 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nqvwl\"" Oct 30 22:08:04.235927 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235850 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 22:08:04.235927 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235856 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qbtfs\"" Oct 30 22:08:04.235927 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235869 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 22:08:04.235927 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235920 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Oct 30 22:08:04.236133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235931 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-m5kkj\"" Oct 30 22:08:04.236133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235939 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Oct 30 22:08:04.236133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235941 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-c4t6x\"" Oct 30 22:08:04.236133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235932 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Oct 30 22:08:04.236133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.235971 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Oct 30 22:08:04.236133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.236092 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 22:08:04.236133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.236121 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-fr6b8\"" Oct 30 22:08:04.236360 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.236138 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 22:08:04.236360 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.236149 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-khd2p\"" Oct 30 22:08:04.236360 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.236186 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9wcbl\"" Oct 30 22:08:04.236360 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.236186 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-f8wq8\"" Oct 30 22:08:04.236360 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.236224 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pcvff\"" Oct 30 22:08:04.236360 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.236248 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 22:08:04.236360 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.236286 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-v75tb\"" Oct 30 22:08:04.244717 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.244699 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 22:08:04.245403 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.245390 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Oct 30 22:08:04.276400 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.276382 4615 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" Oct 30 22:08:04.276492 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.276453 4615 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Oct 30 22:08:04.297511 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.297492 4615 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 22:08:04.297601 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:08:04.297531 4615 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-131-94.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" Oct 30 22:08:04.297601 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.297580 4615 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 22:08:04.297672 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:08:04.297614 4615 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-94.ec2.internal" Oct 30 22:08:04.300738 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.300720 4615 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 22:08:04.342619 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.342599 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-run-openvswitch\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.342715 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.342624 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-cni-netd\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.342715 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.342656 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8rz5\" (UniqueName: \"kubernetes.io/projected/9432e919-3bed-47a1-8b4e-4b979ebdb046-kube-api-access-m8rz5\") pod \"node-ca-h9qqf\" (UID: \"9432e919-3bed-47a1-8b4e-4b979ebdb046\") " pod="openshift-image-registry/node-ca-h9qqf" Oct 30 22:08:04.342715 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.342674 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4-signing-key\") pod \"service-ca-5fffc97bdf-69qgk\" (UID: \"d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4\") " pod="openshift-service-ca/service-ca-5fffc97bdf-69qgk" Oct 30 22:08:04.342831 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.342731 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-socket-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:08:04.342978 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.342955 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-registration-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:08:04.343096 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.342998 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9432e919-3bed-47a1-8b4e-4b979ebdb046-serviceca\") pod \"node-ca-h9qqf\" (UID: \"9432e919-3bed-47a1-8b4e-4b979ebdb046\") " pod="openshift-image-registry/node-ca-h9qqf" Oct 30 22:08:04.343096 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343071 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4-signing-key\") pod \"service-ca-5fffc97bdf-69qgk\" (UID: \"d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4\") " pod="openshift-service-ca/service-ca-5fffc97bdf-69qgk" Oct 30 22:08:04.343096 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343079 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/08904f35-ebda-4220-99c4-b2fb90aa2528-iptables-alerter-script\") pod \"iptables-alerter-zt9w6\" (UID: \"08904f35-ebda-4220-99c4-b2fb90aa2528\") " pod="openshift-network-operator/iptables-alerter-zt9w6" Oct 30 22:08:04.343202 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343113 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b69c485c-88c0-43c5-a69c-d88c0f81fa22-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:08:04.343202 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343138 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-cni-dir\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.343202 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343158 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-cnibin\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.343202 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343177 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-conf-dir\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.343353 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343203 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c135a307-61bc-4e34-9b5c-b97793255db7-tmp-dir\") pod \"node-resolver-q5gw7\" (UID: \"c135a307-61bc-4e34-9b5c-b97793255db7\") " pod="openshift-dns/node-resolver-q5gw7" Oct 30 22:08:04.343353 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343247 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/780de4be-6a4c-4afa-9fee-a3391101d004-crio-socket\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:08:04.343353 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343307 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c135a307-61bc-4e34-9b5c-b97793255db7-tmp-dir\") pod \"node-resolver-q5gw7\" (UID: \"c135a307-61bc-4e34-9b5c-b97793255db7\") " pod="openshift-dns/node-resolver-q5gw7" Oct 30 22:08:04.343488 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343372 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-run-netns\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.343488 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343415 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt5mm\" (UniqueName: \"kubernetes.io/projected/95637308-9cdc-4a32-b573-70e1b4cff546-kube-api-access-xt5mm\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.343488 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343438 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b69c485c-88c0-43c5-a69c-d88c0f81fa22-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:08:04.343488 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343445 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2mlc\" (UniqueName: \"kubernetes.io/projected/80423b86-27d7-42d2-9daa-0399bf2303de-kube-api-access-m2mlc\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.343672 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343472 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/08904f35-ebda-4220-99c4-b2fb90aa2528-iptables-alerter-script\") pod \"iptables-alerter-zt9w6\" (UID: \"08904f35-ebda-4220-99c4-b2fb90aa2528\") " pod="openshift-network-operator/iptables-alerter-zt9w6" Oct 30 22:08:04.343672 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343557 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs\") pod \"network-metrics-daemon-l7k26\" (UID: \"f805f34e-7fc9-430b-9003-16d9765d6657\") " pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:08:04.343672 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343650 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bf28a7e-a8ce-4f33-923a-5d1539978790-tmp-dir\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:08:04.343822 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343679 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b69c485c-88c0-43c5-a69c-d88c0f81fa22-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:08:04.343822 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343734 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7dg5\" (UniqueName: \"kubernetes.io/projected/b69c485c-88c0-43c5-a69c-d88c0f81fa22-kube-api-access-q7dg5\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:08:04.343914 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343851 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3bf28a7e-a8ce-4f33-923a-5d1539978790-tmp-dir\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:08:04.343914 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343889 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:08:04.343994 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343899 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9432e919-3bed-47a1-8b4e-4b979ebdb046-serviceca\") pod \"node-ca-h9qqf\" (UID: \"9432e919-3bed-47a1-8b4e-4b979ebdb046\") " pod="openshift-image-registry/node-ca-h9qqf" Oct 30 22:08:04.343994 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343933 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2kvz\" (UniqueName: \"kubernetes.io/projected/08904f35-ebda-4220-99c4-b2fb90aa2528-kube-api-access-m2kvz\") pod \"iptables-alerter-zt9w6\" (UID: \"08904f35-ebda-4220-99c4-b2fb90aa2528\") " pod="openshift-network-operator/iptables-alerter-zt9w6" Oct 30 22:08:04.343994 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343964 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-kubelet\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.343994 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.343982 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-etc-openvswitch\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.344170 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344010 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-log-socket\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.344170 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344065 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-host\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.344170 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344107 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-dbus\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:08:04.344170 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344139 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-var-lib-kubelet\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.344170 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344166 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-run-systemd\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.344392 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344209 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-run\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.344392 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344208 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-original-pull-secret\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:08:04.344392 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344239 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bf28a7e-a8ce-4f33-923a-5d1539978790-metrics-tls\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:08:04.344392 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344279 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-kubelet-config\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:08:04.344392 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344325 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-cni-binary-copy\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.344392 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344365 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-daemon-config\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.344392 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344391 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-sys-fs\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:08:04.344713 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344415 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/780de4be-6a4c-4afa-9fee-a3391101d004-data-volume\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:08:04.344713 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344453 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-run-ovn-kubernetes\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.344713 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344459 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3bf28a7e-a8ce-4f33-923a-5d1539978790-metrics-tls\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:08:04.344713 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344485 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/95637308-9cdc-4a32-b573-70e1b4cff546-ovnkube-script-lib\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.344713 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344524 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-sysctl-d\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.344713 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344523 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/780de4be-6a4c-4afa-9fee-a3391101d004-data-volume\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:08:04.344713 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344559 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04f40ed9-0eef-4946-b3e6-feafcfee8e03-service-ca-bundle\") pod \"periodic-gathering-r9szz-8wv8c\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:08:04.344713 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344563 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-cni-binary-copy\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.344713 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344600 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/08904f35-ebda-4220-99c4-b2fb90aa2528-host-slash\") pod \"iptables-alerter-zt9w6\" (UID: \"08904f35-ebda-4220-99c4-b2fb90aa2528\") " pod="openshift-network-operator/iptables-alerter-zt9w6" Oct 30 22:08:04.344713 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344621 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-etc-kubernetes\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.344713 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344639 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/12b2045b-a302-47a1-b039-b8eadcc50c61-konnectivity-ca\") pod \"konnectivity-agent-6qz5h\" (UID: \"12b2045b-a302-47a1-b039-b8eadcc50c61\") " pod="kube-system/konnectivity-agent-6qz5h" Oct 30 22:08:04.344713 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344649 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-daemon-config\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.344713 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344657 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxhfx\" (UniqueName: \"kubernetes.io/projected/749dd349-e1d9-4b55-ab5c-5a30739a8cae-kube-api-access-gxhfx\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:08:04.344713 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344698 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-node-log\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.345430 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344744 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/95637308-9cdc-4a32-b573-70e1b4cff546-ovn-node-metrics-cert\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.345430 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.344793 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-kubernetes\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.345635 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.345583 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-sysctl-conf\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.345635 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.345619 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/95637308-9cdc-4a32-b573-70e1b4cff546-ovnkube-script-lib\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.345810 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.345649 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/12b2045b-a302-47a1-b039-b8eadcc50c61-konnectivity-ca\") pod \"konnectivity-agent-6qz5h\" (UID: \"12b2045b-a302-47a1-b039-b8eadcc50c61\") " pod="kube-system/konnectivity-agent-6qz5h" Oct 30 22:08:04.345810 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.345666 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wrvl\" (UniqueName: \"kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl\") pod \"network-check-target-pmkb2\" (UID: \"d3d3d52b-f209-4ad4-b531-4fe19bad4bf1\") " pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:08:04.345810 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.345734 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/780de4be-6a4c-4afa-9fee-a3391101d004-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:08:04.345810 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.345741 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04f40ed9-0eef-4946-b3e6-feafcfee8e03-service-ca-bundle\") pod \"periodic-gathering-r9szz-8wv8c\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:08:04.345810 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.345803 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwrm9\" (UniqueName: \"kubernetes.io/projected/c1f53f6e-aadc-41fd-aa64-20a466fb1769-kube-api-access-kwrm9\") pod \"migrator-66df5d496b-lc6lf\" (UID: \"c1f53f6e-aadc-41fd-aa64-20a466fb1769\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-lc6lf" Oct 30 22:08:04.346025 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.345935 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/95637308-9cdc-4a32-b573-70e1b4cff546-ovn-node-metrics-cert\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.346086 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.345862 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-sysconfig\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.346086 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.346063 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/780de4be-6a4c-4afa-9fee-a3391101d004-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:08:04.346180 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.346084 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-sys\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.346180 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.346142 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6scx6\" (UniqueName: \"kubernetes.io/projected/3bf28a7e-a8ce-4f33-923a-5d1539978790-kube-api-access-6scx6\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:08:04.346273 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.346206 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b69c485c-88c0-43c5-a69c-d88c0f81fa22-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:08:04.346273 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.346243 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-system-cni-dir\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.346358 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.346280 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgtzk\" (UniqueName: \"kubernetes.io/projected/780de4be-6a4c-4afa-9fee-a3391101d004-kube-api-access-xgtzk\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:08:04.346358 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.346314 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-var-lib-openvswitch\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.346430 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.346396 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b69c485c-88c0-43c5-a69c-d88c0f81fa22-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:08:04.346549 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.346518 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/95637308-9cdc-4a32-b573-70e1b4cff546-ovnkube-config\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.346617 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.346594 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-socket-dir-parent\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.346666 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.346633 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-run-ovn\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.346736 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.346701 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.346805 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.346786 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-lib-modules\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.346855 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.346825 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l28gx\" (UniqueName: \"kubernetes.io/projected/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4-kube-api-access-l28gx\") pod \"service-ca-5fffc97bdf-69qgk\" (UID: \"d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4\") " pod="openshift-service-ca/service-ca-5fffc97bdf-69qgk" Oct 30 22:08:04.346948 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.346928 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/04f40ed9-0eef-4946-b3e6-feafcfee8e03-archives-path\") pod \"periodic-gathering-r9szz-8wv8c\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:08:04.347005 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.346921 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/95637308-9cdc-4a32-b573-70e1b4cff546-ovnkube-config\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.347005 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.346980 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b69c485c-88c0-43c5-a69c-d88c0f81fa22-system-cni-dir\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:08:04.347158 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.347141 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/04f40ed9-0eef-4946-b3e6-feafcfee8e03-archives-path\") pod \"periodic-gathering-r9szz-8wv8c\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:08:04.347216 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.347180 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-var-lib-cni-bin\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.347260 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.347240 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:08:04.347309 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.347276 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/95637308-9cdc-4a32-b573-70e1b4cff546-env-overrides\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.347358 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.347321 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szj9w\" (UniqueName: \"kubernetes.io/projected/04f40ed9-0eef-4946-b3e6-feafcfee8e03-kube-api-access-szj9w\") pod \"periodic-gathering-r9szz-8wv8c\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:08:04.347402 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.347360 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-run-netns\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.347402 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.347391 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-hostroot\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.347493 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.347423 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/12b2045b-a302-47a1-b039-b8eadcc50c61-agent-certs\") pod \"konnectivity-agent-6qz5h\" (UID: \"12b2045b-a302-47a1-b039-b8eadcc50c61\") " pod="kube-system/konnectivity-agent-6qz5h" Oct 30 22:08:04.347541 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.347498 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-cni-bin\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.347541 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.347526 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9432e919-3bed-47a1-8b4e-4b979ebdb046-host\") pod \"node-ca-h9qqf\" (UID: \"9432e919-3bed-47a1-8b4e-4b979ebdb046\") " pod="openshift-image-registry/node-ca-h9qqf" Oct 30 22:08:04.347627 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.347553 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-modprobe-d\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.347627 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.347581 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-var-lib-kubelet\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.347627 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.347610 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80423b86-27d7-42d2-9daa-0399bf2303de-tmp\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.347764 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.347643 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3bf28a7e-a8ce-4f33-923a-5d1539978790-config-volume\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:08:04.347764 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.347675 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4-signing-cabundle\") pod \"service-ca-5fffc97bdf-69qgk\" (UID: \"d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4\") " pod="openshift-service-ca/service-ca-5fffc97bdf-69qgk" Oct 30 22:08:04.347849 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.347794 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b69c485c-88c0-43c5-a69c-d88c0f81fa22-cnibin\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:08:04.347849 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.347834 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b69c485c-88c0-43c5-a69c-d88c0f81fa22-cni-binary-copy\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:08:04.347976 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.347861 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-run-k8s-cni-cncf-io\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.347976 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.347897 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-run-multus-certs\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.347976 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.347925 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:08:04.347976 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.347952 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-slash\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.347976 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.347976 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/80423b86-27d7-42d2-9daa-0399bf2303de-etc-tuned\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.348259 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.348017 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scwfv\" (UniqueName: \"kubernetes.io/projected/f805f34e-7fc9-430b-9003-16d9765d6657-kube-api-access-scwfv\") pod \"network-metrics-daemon-l7k26\" (UID: \"f805f34e-7fc9-430b-9003-16d9765d6657\") " pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:08:04.348259 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.348057 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04f40ed9-0eef-4946-b3e6-feafcfee8e03-trusted-ca-bundle\") pod \"periodic-gathering-r9szz-8wv8c\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:08:04.348259 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.348086 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24e7e5b3-a4ea-4322-915b-5694ea1f3112-cert\") pod \"ingress-canary-khpnv\" (UID: \"24e7e5b3-a4ea-4322-915b-5694ea1f3112\") " pod="openshift-ingress-canary/ingress-canary-khpnv" Oct 30 22:08:04.348259 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.348116 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pxnh\" (UniqueName: \"kubernetes.io/projected/24e7e5b3-a4ea-4322-915b-5694ea1f3112-kube-api-access-4pxnh\") pod \"ingress-canary-khpnv\" (UID: \"24e7e5b3-a4ea-4322-915b-5694ea1f3112\") " pod="openshift-ingress-canary/ingress-canary-khpnv" Oct 30 22:08:04.348259 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.348144 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c135a307-61bc-4e34-9b5c-b97793255db7-hosts-file\") pod \"node-resolver-q5gw7\" (UID: \"c135a307-61bc-4e34-9b5c-b97793255db7\") " pod="openshift-dns/node-resolver-q5gw7" Oct 30 22:08:04.348259 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.348173 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-etc-selinux\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:08:04.348259 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.348198 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-systemd-units\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.348259 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.348226 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-var-lib-cni-multus\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.348685 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.348399 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffjdk\" (UniqueName: \"kubernetes.io/projected/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-kube-api-access-ffjdk\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.348685 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.348429 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z8f9\" (UniqueName: \"kubernetes.io/projected/c135a307-61bc-4e34-9b5c-b97793255db7-kube-api-access-8z8f9\") pod \"node-resolver-q5gw7\" (UID: \"c135a307-61bc-4e34-9b5c-b97793255db7\") " pod="openshift-dns/node-resolver-q5gw7" Oct 30 22:08:04.348685 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.348461 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-device-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:08:04.348685 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.348485 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-systemd\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.348908 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.348740 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4-signing-cabundle\") pod \"service-ca-5fffc97bdf-69qgk\" (UID: \"d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4\") " pod="openshift-service-ca/service-ca-5fffc97bdf-69qgk" Oct 30 22:08:04.349070 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.348973 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/12b2045b-a302-47a1-b039-b8eadcc50c61-agent-certs\") pod \"konnectivity-agent-6qz5h\" (UID: \"12b2045b-a302-47a1-b039-b8eadcc50c61\") " pod="kube-system/konnectivity-agent-6qz5h" Oct 30 22:08:04.349223 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.349146 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/95637308-9cdc-4a32-b573-70e1b4cff546-env-overrides\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.349453 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.349438 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80423b86-27d7-42d2-9daa-0399bf2303de-tmp\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.349705 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.349598 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/80423b86-27d7-42d2-9daa-0399bf2303de-etc-tuned\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.349857 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.349830 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3bf28a7e-a8ce-4f33-923a-5d1539978790-config-volume\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:08:04.349912 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.349859 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b69c485c-88c0-43c5-a69c-d88c0f81fa22-cni-binary-copy\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:08:04.349995 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.349979 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24e7e5b3-a4ea-4322-915b-5694ea1f3112-cert\") pod \"ingress-canary-khpnv\" (UID: \"24e7e5b3-a4ea-4322-915b-5694ea1f3112\") " pod="openshift-ingress-canary/ingress-canary-khpnv" Oct 30 22:08:04.350838 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.350819 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04f40ed9-0eef-4946-b3e6-feafcfee8e03-trusted-ca-bundle\") pod \"periodic-gathering-r9szz-8wv8c\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:08:04.362240 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.362214 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8rz5\" (UniqueName: \"kubernetes.io/projected/9432e919-3bed-47a1-8b4e-4b979ebdb046-kube-api-access-m8rz5\") pod \"node-ca-h9qqf\" (UID: \"9432e919-3bed-47a1-8b4e-4b979ebdb046\") " pod="openshift-image-registry/node-ca-h9qqf" Oct 30 22:08:04.362599 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.362582 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxhfx\" (UniqueName: \"kubernetes.io/projected/749dd349-e1d9-4b55-ab5c-5a30739a8cae-kube-api-access-gxhfx\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:08:04.363279 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.363142 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2kvz\" (UniqueName: \"kubernetes.io/projected/08904f35-ebda-4220-99c4-b2fb90aa2528-kube-api-access-m2kvz\") pod \"iptables-alerter-zt9w6\" (UID: \"08904f35-ebda-4220-99c4-b2fb90aa2528\") " pod="openshift-network-operator/iptables-alerter-zt9w6" Oct 30 22:08:04.363279 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.363224 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7dg5\" (UniqueName: \"kubernetes.io/projected/b69c485c-88c0-43c5-a69c-d88c0f81fa22-kube-api-access-q7dg5\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:08:04.363423 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.363357 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6scx6\" (UniqueName: \"kubernetes.io/projected/3bf28a7e-a8ce-4f33-923a-5d1539978790-kube-api-access-6scx6\") pod \"dns-default-xtbrk\" (UID: \"3bf28a7e-a8ce-4f33-923a-5d1539978790\") " pod="openshift-dns/dns-default-xtbrk" Oct 30 22:08:04.363423 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.363366 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgtzk\" (UniqueName: \"kubernetes.io/projected/780de4be-6a4c-4afa-9fee-a3391101d004-kube-api-access-xgtzk\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:08:04.363423 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.363392 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wrvl\" (UniqueName: \"kubernetes.io/projected/d3d3d52b-f209-4ad4-b531-4fe19bad4bf1-kube-api-access-4wrvl\") pod \"network-check-target-pmkb2\" (UID: \"d3d3d52b-f209-4ad4-b531-4fe19bad4bf1\") " pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:08:04.363576 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.363550 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2mlc\" (UniqueName: \"kubernetes.io/projected/80423b86-27d7-42d2-9daa-0399bf2303de-kube-api-access-m2mlc\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.363624 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.363559 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwrm9\" (UniqueName: \"kubernetes.io/projected/c1f53f6e-aadc-41fd-aa64-20a466fb1769-kube-api-access-kwrm9\") pod \"migrator-66df5d496b-lc6lf\" (UID: \"c1f53f6e-aadc-41fd-aa64-20a466fb1769\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-lc6lf" Oct 30 22:08:04.363819 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.363802 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l28gx\" (UniqueName: \"kubernetes.io/projected/d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4-kube-api-access-l28gx\") pod \"service-ca-5fffc97bdf-69qgk\" (UID: \"d6a0f8e7-7e6d-46d2-97bc-94ce55deb6d4\") " pod="openshift-service-ca/service-ca-5fffc97bdf-69qgk" Oct 30 22:08:04.363883 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.363818 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt5mm\" (UniqueName: \"kubernetes.io/projected/95637308-9cdc-4a32-b573-70e1b4cff546-kube-api-access-xt5mm\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.364239 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.364221 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scwfv\" (UniqueName: \"kubernetes.io/projected/f805f34e-7fc9-430b-9003-16d9765d6657-kube-api-access-scwfv\") pod \"network-metrics-daemon-l7k26\" (UID: \"f805f34e-7fc9-430b-9003-16d9765d6657\") " pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:08:04.364980 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.364962 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szj9w\" (UniqueName: \"kubernetes.io/projected/04f40ed9-0eef-4946-b3e6-feafcfee8e03-kube-api-access-szj9w\") pod \"periodic-gathering-r9szz-8wv8c\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:08:04.365064 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.365012 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pxnh\" (UniqueName: \"kubernetes.io/projected/24e7e5b3-a4ea-4322-915b-5694ea1f3112-kube-api-access-4pxnh\") pod \"ingress-canary-khpnv\" (UID: \"24e7e5b3-a4ea-4322-915b-5694ea1f3112\") " pod="openshift-ingress-canary/ingress-canary-khpnv" Oct 30 22:08:04.365109 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.365062 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffjdk\" (UniqueName: \"kubernetes.io/projected/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-kube-api-access-ffjdk\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.365253 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.365238 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z8f9\" (UniqueName: \"kubernetes.io/projected/c135a307-61bc-4e34-9b5c-b97793255db7-kube-api-access-8z8f9\") pod \"node-resolver-q5gw7\" (UID: \"c135a307-61bc-4e34-9b5c-b97793255db7\") " pod="openshift-dns/node-resolver-q5gw7" Oct 30 22:08:04.449077 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.448980 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-run-openvswitch\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.449077 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449014 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-cni-netd\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.449077 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449035 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-socket-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:08:04.449345 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449097 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-registration-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:08:04.449345 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449114 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-cni-netd\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.449345 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449140 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-socket-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:08:04.449345 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449123 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-run-openvswitch\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.449345 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449121 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-cni-dir\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.449345 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449178 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-registration-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:08:04.449345 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449176 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-cni-dir\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.449345 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449208 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-cnibin\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.449345 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449253 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-cnibin\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.449345 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449276 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-conf-dir\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.449345 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449303 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/780de4be-6a4c-4afa-9fee-a3391101d004-crio-socket\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:08:04.449345 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449351 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-conf-dir\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449327 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-run-netns\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449373 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/780de4be-6a4c-4afa-9fee-a3391101d004-crio-socket\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449382 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-run-netns\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449386 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs\") pod \"network-metrics-daemon-l7k26\" (UID: \"f805f34e-7fc9-430b-9003-16d9765d6657\") " pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449414 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b69c485c-88c0-43c5-a69c-d88c0f81fa22-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449444 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-kubelet\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449468 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-etc-openvswitch\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449491 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-log-socket\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449495 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b69c485c-88c0-43c5-a69c-d88c0f81fa22-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449516 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-host\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449526 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-kubelet\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449537 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-etc-openvswitch\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449541 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-dbus\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449550 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-log-socket\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449573 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-var-lib-kubelet\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449583 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-host\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449591 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-dbus\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449599 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-run-systemd\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449620 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-var-lib-kubelet\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449623 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-run\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449646 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-kubelet-config\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449651 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-run-systemd\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449674 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-sys-fs\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449698 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-run-ovn-kubernetes\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449724 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-sysctl-d\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449724 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9-kubelet-config\") pod \"global-pull-secret-syncer-gsvm9\" (UID: \"bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9\") " pod="kube-system/global-pull-secret-syncer-gsvm9" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449743 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-run\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449729 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-sys-fs\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449752 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-run-ovn-kubernetes\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449756 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/08904f35-ebda-4220-99c4-b2fb90aa2528-host-slash\") pod \"iptables-alerter-zt9w6\" (UID: \"08904f35-ebda-4220-99c4-b2fb90aa2528\") " pod="openshift-network-operator/iptables-alerter-zt9w6" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449760 4615 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 22:08:04.449830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449845 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/08904f35-ebda-4220-99c4-b2fb90aa2528-host-slash\") pod \"iptables-alerter-zt9w6\" (UID: \"08904f35-ebda-4220-99c4-b2fb90aa2528\") " pod="openshift-network-operator/iptables-alerter-zt9w6" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449882 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-sysctl-d\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449884 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-etc-kubernetes\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449926 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-node-log\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449900 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-etc-kubernetes\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449948 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-kubernetes\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449992 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-sysctl-conf\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.449994 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-node-log\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450061 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-sysconfig\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450078 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-kubernetes\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450080 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-sys\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450113 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-sys\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450117 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-sysconfig\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450113 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-system-cni-dir\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450149 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-var-lib-openvswitch\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450149 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-system-cni-dir\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450170 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-socket-dir-parent\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450173 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-sysctl-conf\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450193 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-run-ovn\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450194 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-var-lib-openvswitch\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450211 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450218 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-multus-socket-dir-parent\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450229 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-lib-modules\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450259 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b69c485c-88c0-43c5-a69c-d88c0f81fa22-system-cni-dir\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450299 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-run-ovn\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450295 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-var-lib-cni-bin\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450301 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450318 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b69c485c-88c0-43c5-a69c-d88c0f81fa22-system-cni-dir\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450326 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450358 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-var-lib-cni-bin\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450377 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-run-netns\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450327 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-lib-modules\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:08:04.450401 4615 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450406 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-run-netns\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450424 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-hostroot\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450460 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-hostroot\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450465 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-cni-bin\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:08:04.450475 4615 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls podName:780de4be-6a4c-4afa-9fee-a3391101d004 nodeName:}" failed. No retries permitted until 2026-10-30 22:08:04.950460431 +0000 UTC m=+1.866146717 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gs6ts" (UID: "780de4be-6a4c-4afa-9fee-a3391101d004") : secret "insights-runtime-extractor-tls" not found Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450502 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9432e919-3bed-47a1-8b4e-4b979ebdb046-host\") pod \"node-ca-h9qqf\" (UID: \"9432e919-3bed-47a1-8b4e-4b979ebdb046\") " pod="openshift-image-registry/node-ca-h9qqf" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450502 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-cni-bin\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450534 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9432e919-3bed-47a1-8b4e-4b979ebdb046-host\") pod \"node-ca-h9qqf\" (UID: \"9432e919-3bed-47a1-8b4e-4b979ebdb046\") " pod="openshift-image-registry/node-ca-h9qqf" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450525 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-modprobe-d\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450576 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-var-lib-kubelet\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450609 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-modprobe-d\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450606 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b69c485c-88c0-43c5-a69c-d88c0f81fa22-cnibin\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450645 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b69c485c-88c0-43c5-a69c-d88c0f81fa22-cnibin\") pod \"multus-additional-cni-plugins-lzh8c\" (UID: \"b69c485c-88c0-43c5-a69c-d88c0f81fa22\") " pod="openshift-multus/multus-additional-cni-plugins-lzh8c" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450650 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-run-k8s-cni-cncf-io\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450673 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-run-multus-certs\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450677 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-var-lib-kubelet\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450688 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450714 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-slash\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450721 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-run-k8s-cni-cncf-io\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450724 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-run-multus-certs\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450744 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c135a307-61bc-4e34-9b5c-b97793255db7-hosts-file\") pod \"node-resolver-q5gw7\" (UID: \"c135a307-61bc-4e34-9b5c-b97793255db7\") " pod="openshift-dns/node-resolver-q5gw7" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450749 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450779 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-host-slash\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450801 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-etc-selinux\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450827 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-systemd-units\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450844 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-var-lib-cni-multus\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450802 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c135a307-61bc-4e34-9b5c-b97793255db7-hosts-file\") pod \"node-resolver-q5gw7\" (UID: \"c135a307-61bc-4e34-9b5c-b97793255db7\") " pod="openshift-dns/node-resolver-q5gw7" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450858 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-etc-selinux\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450862 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-device-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450894 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/749dd349-e1d9-4b55-ab5c-5a30739a8cae-device-dir\") pod \"aws-ebs-csi-driver-node-bmr79\" (UID: \"749dd349-e1d9-4b55-ab5c-5a30739a8cae\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bmr79" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450898 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-systemd\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450906 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9de98c42-a20f-4d6f-b8ab-d0cabf70dedf-host-var-lib-cni-multus\") pod \"multus-4gdsr\" (UID: \"9de98c42-a20f-4d6f-b8ab-d0cabf70dedf\") " pod="openshift-multus/multus-4gdsr" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450899 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/95637308-9cdc-4a32-b573-70e1b4cff546-systemd-units\") pod \"ovnkube-node-wqj4x\" (UID: \"95637308-9cdc-4a32-b573-70e1b4cff546\") " pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:04.451133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.450961 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/80423b86-27d7-42d2-9daa-0399bf2303de-etc-systemd\") pod \"tuned-w692c\" (UID: \"80423b86-27d7-42d2-9daa-0399bf2303de\") " pod="openshift-cluster-node-tuning-operator/tuned-w692c" Oct 30 22:08:04.453797 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.452014 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f805f34e-7fc9-430b-9003-16d9765d6657-metrics-certs\") pod \"network-metrics-daemon-l7k26\" (UID: \"f805f34e-7fc9-430b-9003-16d9765d6657\") " pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:08:04.513631 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.513602 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-l7k26" Oct 30 22:08:04.521908 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.521888 4615 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 22:08:04.713345 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.713265 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:04.832258 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.832229 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-l7k26"] Oct 30 22:08:04.833789 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.833764 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:04.835707 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:04.835674 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf805f34e_7fc9_430b_9003_16d9765d6657.slice/crio-b7aae275b3f3fd9f9b7a9f16c43564dfeaedcbcf2dc0869db51e91e8cabf4e74 WatchSource:0}: Error finding container b7aae275b3f3fd9f9b7a9f16c43564dfeaedcbcf2dc0869db51e91e8cabf4e74: Status 404 returned error can't find the container with id b7aae275b3f3fd9f9b7a9f16c43564dfeaedcbcf2dc0869db51e91e8cabf4e74 Oct 30 22:08:04.955253 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:04.955220 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:08:04.955394 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:08:04.955364 4615 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 22:08:04.955436 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:08:04.955420 4615 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls podName:780de4be-6a4c-4afa-9fee-a3391101d004 nodeName:}" failed. No retries permitted until 2026-10-30 22:08:05.955406267 +0000 UTC m=+2.871092553 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gs6ts" (UID: "780de4be-6a4c-4afa-9fee-a3391101d004") : secret "insights-runtime-extractor-tls" not found Oct 30 22:08:05.242489 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:05.242457 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:05.282236 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:05.282184 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xtbrk" event={"ID":"3bf28a7e-a8ce-4f33-923a-5d1539978790","Type":"ContainerStarted","Data":"00eef08d03077c079f0fc1482731a3ce629ba3ee59abfa9bf95a4065d7859054"} Oct 30 22:08:05.283510 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:05.283484 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l7k26" event={"ID":"f805f34e-7fc9-430b-9003-16d9765d6657","Type":"ContainerStarted","Data":"b7aae275b3f3fd9f9b7a9f16c43564dfeaedcbcf2dc0869db51e91e8cabf4e74"} Oct 30 22:08:05.436007 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:05.435948 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:05.633436 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:05.633402 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:05.667103 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:05.667076 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-6qz5h" Oct 30 22:08:05.963083 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:05.962998 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:08:05.963210 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:08:05.963125 4615 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 22:08:05.963210 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:08:05.963177 4615 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls podName:780de4be-6a4c-4afa-9fee-a3391101d004 nodeName:}" failed. No retries permitted until 2026-10-30 22:08:07.963163797 +0000 UTC m=+4.878850077 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gs6ts" (UID: "780de4be-6a4c-4afa-9fee-a3391101d004") : secret "insights-runtime-extractor-tls" not found Oct 30 22:08:06.038256 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:06.038205 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:06.288895 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:06.288857 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xtbrk" event={"ID":"3bf28a7e-a8ce-4f33-923a-5d1539978790","Type":"ContainerStarted","Data":"2a71dadb45ee469f5923c8d1eb0143ceb347459a0717221dae503024a84f060d"} Oct 30 22:08:06.289267 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:06.288964 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xtbrk" Oct 30 22:08:06.290361 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:06.290339 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l7k26" event={"ID":"f805f34e-7fc9-430b-9003-16d9765d6657","Type":"ContainerStarted","Data":"f38652367a7d2a1b6fb41f5f793246376b2a4cba84b458c42287e13e4d674392"} Oct 30 22:08:06.836635 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:06.836586 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:07.215273 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:07.215248 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:07.234863 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:07.234835 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:07.295934 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:07.295903 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-l7k26" event={"ID":"f805f34e-7fc9-430b-9003-16d9765d6657","Type":"ContainerStarted","Data":"8604659f1ed8a6b7ee4c173b4aa50bc425f3373ec69461b6acdbe81e3aa0bf89"} Oct 30 22:08:07.433060 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:07.433024 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:07.547172 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:07.547090 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:07.666567 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:07.666537 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:07.979210 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:07.979176 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:08:07.979406 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:08:07.979290 4615 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 22:08:07.979406 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:08:07.979353 4615 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls podName:780de4be-6a4c-4afa-9fee-a3391101d004 nodeName:}" failed. No retries permitted until 2026-10-30 22:08:11.979340156 +0000 UTC m=+8.895026438 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gs6ts" (UID: "780de4be-6a4c-4afa-9fee-a3391101d004") : secret "insights-runtime-extractor-tls" not found Oct 30 22:08:08.030992 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:08.030959 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:08.853530 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:08.853488 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:08.869438 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:08.869420 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:08:09.233165 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:09.233132 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:09.329811 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:09.329783 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:09.330016 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:09.329999 4615 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 22:08:09.345635 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:09.345609 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:09.447300 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:09.447267 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:09.627414 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:09.627328 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:10.037056 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:10.037016 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:10.243949 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:10.243913 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:11.477003 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:11.476966 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:11.580032 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:11.579999 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:12.005610 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:12.005570 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:08:12.007823 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:12.007800 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/780de4be-6a4c-4afa-9fee-a3391101d004-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gs6ts\" (UID: \"780de4be-6a4c-4afa-9fee-a3391101d004\") " pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:08:12.020809 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:12.020783 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gs6ts" Oct 30 22:08:12.311889 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:12.311802 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gs6ts"] Oct 30 22:08:12.315429 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:12.315395 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod780de4be_6a4c_4afa_9fee_a3391101d004.slice/crio-87ae30f9e44ab6b654270e7c1b6d8ac140d45cffa3d8b020b899d6842c6e83ad WatchSource:0}: Error finding container 87ae30f9e44ab6b654270e7c1b6d8ac140d45cffa3d8b020b899d6842c6e83ad: Status 404 returned error can't find the container with id 87ae30f9e44ab6b654270e7c1b6d8ac140d45cffa3d8b020b899d6842c6e83ad Oct 30 22:08:13.316412 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:13.316308 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gs6ts" event={"ID":"780de4be-6a4c-4afa-9fee-a3391101d004","Type":"ContainerStarted","Data":"bb762e2985d24f6809b5be259313d86d36c78203c46855ff800d56ce8cad0476"} Oct 30 22:08:13.316412 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:13.316386 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gs6ts" event={"ID":"780de4be-6a4c-4afa-9fee-a3391101d004","Type":"ContainerStarted","Data":"39b51d3ea21552e6c7725d69cb04c8ec92340decf5ea377a4ec233a1d8f78650"} Oct 30 22:08:13.316872 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:13.316417 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gs6ts" event={"ID":"780de4be-6a4c-4afa-9fee-a3391101d004","Type":"ContainerStarted","Data":"87ae30f9e44ab6b654270e7c1b6d8ac140d45cffa3d8b020b899d6842c6e83ad"} Oct 30 22:08:14.321228 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:14.321194 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gs6ts" event={"ID":"780de4be-6a4c-4afa-9fee-a3391101d004","Type":"ContainerStarted","Data":"f53b1d460d832235dbaae0b264c7a218793d341c4ae46ec45a9c839e292d7928"} Oct 30 22:08:14.766013 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:14.765973 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v9mj4"] Oct 30 22:08:14.769373 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:14.769350 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v9mj4" Oct 30 22:08:14.790088 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:14.790038 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v9mj4"] Oct 30 22:08:14.799621 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:14.799596 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Oct 30 22:08:14.799771 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:14.799635 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-6dzfs\"" Oct 30 22:08:14.924403 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:14.924371 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9ba0eb49-d6b6-4b0b-b86c-97245a119b2c-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-v9mj4\" (UID: \"9ba0eb49-d6b6-4b0b-b86c-97245a119b2c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v9mj4" Oct 30 22:08:14.947663 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:14.947610 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:15.025394 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:15.025303 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9ba0eb49-d6b6-4b0b-b86c-97245a119b2c-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-v9mj4\" (UID: \"9ba0eb49-d6b6-4b0b-b86c-97245a119b2c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v9mj4" Oct 30 22:08:15.027915 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:15.027889 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9ba0eb49-d6b6-4b0b-b86c-97245a119b2c-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-v9mj4\" (UID: \"9ba0eb49-d6b6-4b0b-b86c-97245a119b2c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v9mj4" Oct 30 22:08:15.063574 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:15.063536 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:15.081998 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:15.081964 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v9mj4" Oct 30 22:08:15.175185 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:15.175153 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:15.369348 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:15.369259 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v9mj4"] Oct 30 22:08:15.373152 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:15.373121 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ba0eb49_d6b6_4b0b_b86c_97245a119b2c.slice/crio-69ac7e061a892896cc17edd1fdd040d6b7059335eab9e3fdde6bd9ba8b2fdf4f WatchSource:0}: Error finding container 69ac7e061a892896cc17edd1fdd040d6b7059335eab9e3fdde6bd9ba8b2fdf4f: Status 404 returned error can't find the container with id 69ac7e061a892896cc17edd1fdd040d6b7059335eab9e3fdde6bd9ba8b2fdf4f Oct 30 22:08:16.298500 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:16.298476 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xtbrk" Oct 30 22:08:16.330174 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:16.330136 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v9mj4" event={"ID":"9ba0eb49-d6b6-4b0b-b86c-97245a119b2c","Type":"ContainerStarted","Data":"69ac7e061a892896cc17edd1fdd040d6b7059335eab9e3fdde6bd9ba8b2fdf4f"} Oct 30 22:08:17.334737 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:17.334689 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v9mj4" event={"ID":"9ba0eb49-d6b6-4b0b-b86c-97245a119b2c","Type":"ContainerStarted","Data":"30bf12fed80a56762461aa1b7b6b76f113c1f487d4ac623d762fc2a396a80d0a"} Oct 30 22:08:17.335258 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:17.335019 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v9mj4" Oct 30 22:08:17.339823 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:17.339801 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v9mj4" Oct 30 22:08:17.370237 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:17.370188 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-v9mj4" podStartSLOduration=2.453895547 podStartE2EDuration="3.37017433s" totalImagesPullingTime="916.278783ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:17.369701783 +0000 UTC m=+14.285388085" watchObservedRunningTime="2026-10-30 22:08:17.37017433 +0000 UTC m=+14.285860632" Oct 30 22:08:18.117745 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.117708 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:18.207182 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.207152 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-69twj"] Oct 30 22:08:18.211519 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.211492 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-69twj" Oct 30 22:08:18.238291 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.238259 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 22:08:18.238469 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.238382 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Oct 30 22:08:18.238469 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.238402 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 22:08:18.238469 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.238384 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Oct 30 22:08:18.238602 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.238465 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-qnjtd\"" Oct 30 22:08:18.239093 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.239075 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-69twj"] Oct 30 22:08:18.239950 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.239925 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:18.248893 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.248874 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 30 22:08:18.348849 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.348812 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/93e3dae5-1990-4737-91ae-178d25c4ff1d-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-69twj\" (UID: \"93e3dae5-1990-4737-91ae-178d25c4ff1d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-69twj" Oct 30 22:08:18.349336 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.348860 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93e3dae5-1990-4737-91ae-178d25c4ff1d-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-69twj\" (UID: \"93e3dae5-1990-4737-91ae-178d25c4ff1d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-69twj" Oct 30 22:08:18.349336 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.348883 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vzr4\" (UniqueName: \"kubernetes.io/projected/93e3dae5-1990-4737-91ae-178d25c4ff1d-kube-api-access-9vzr4\") pod \"prometheus-operator-9cd89b74c-69twj\" (UID: \"93e3dae5-1990-4737-91ae-178d25c4ff1d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-69twj" Oct 30 22:08:18.349336 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.348939 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93e3dae5-1990-4737-91ae-178d25c4ff1d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-69twj\" (UID: \"93e3dae5-1990-4737-91ae-178d25c4ff1d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-69twj" Oct 30 22:08:18.360653 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.360626 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:18.449883 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.449845 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93e3dae5-1990-4737-91ae-178d25c4ff1d-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-69twj\" (UID: \"93e3dae5-1990-4737-91ae-178d25c4ff1d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-69twj" Oct 30 22:08:18.449883 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.449886 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vzr4\" (UniqueName: \"kubernetes.io/projected/93e3dae5-1990-4737-91ae-178d25c4ff1d-kube-api-access-9vzr4\") pod \"prometheus-operator-9cd89b74c-69twj\" (UID: \"93e3dae5-1990-4737-91ae-178d25c4ff1d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-69twj" Oct 30 22:08:18.450128 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.449917 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93e3dae5-1990-4737-91ae-178d25c4ff1d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-69twj\" (UID: \"93e3dae5-1990-4737-91ae-178d25c4ff1d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-69twj" Oct 30 22:08:18.450128 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.449981 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/93e3dae5-1990-4737-91ae-178d25c4ff1d-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-69twj\" (UID: \"93e3dae5-1990-4737-91ae-178d25c4ff1d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-69twj" Oct 30 22:08:18.450670 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.450648 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93e3dae5-1990-4737-91ae-178d25c4ff1d-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-69twj\" (UID: \"93e3dae5-1990-4737-91ae-178d25c4ff1d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-69twj" Oct 30 22:08:18.452433 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.452412 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/93e3dae5-1990-4737-91ae-178d25c4ff1d-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-69twj\" (UID: \"93e3dae5-1990-4737-91ae-178d25c4ff1d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-69twj" Oct 30 22:08:18.452541 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.452518 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93e3dae5-1990-4737-91ae-178d25c4ff1d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-69twj\" (UID: \"93e3dae5-1990-4737-91ae-178d25c4ff1d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-69twj" Oct 30 22:08:18.467771 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.467744 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vzr4\" (UniqueName: \"kubernetes.io/projected/93e3dae5-1990-4737-91ae-178d25c4ff1d-kube-api-access-9vzr4\") pod \"prometheus-operator-9cd89b74c-69twj\" (UID: \"93e3dae5-1990-4737-91ae-178d25c4ff1d\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-69twj" Oct 30 22:08:18.524567 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.524532 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-69twj" Oct 30 22:08:18.818965 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:18.818876 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-69twj"] Oct 30 22:08:18.823170 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:18.823140 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93e3dae5_1990_4737_91ae_178d25c4ff1d.slice/crio-156cf806bb0e029c3fec3759977a2da6977461c8d0456a112f36ea31b71a08e8 WatchSource:0}: Error finding container 156cf806bb0e029c3fec3759977a2da6977461c8d0456a112f36ea31b71a08e8: Status 404 returned error can't find the container with id 156cf806bb0e029c3fec3759977a2da6977461c8d0456a112f36ea31b71a08e8 Oct 30 22:08:19.349102 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:19.349062 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-69twj" event={"ID":"93e3dae5-1990-4737-91ae-178d25c4ff1d","Type":"ContainerStarted","Data":"156cf806bb0e029c3fec3759977a2da6977461c8d0456a112f36ea31b71a08e8"} Oct 30 22:08:20.354240 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:20.354161 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-69twj" event={"ID":"93e3dae5-1990-4737-91ae-178d25c4ff1d","Type":"ContainerStarted","Data":"e814d1a85ceb6a12d468a7ea5aae2293c307b109f7569dcdf3fe040b190876b7"} Oct 30 22:08:20.354240 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:20.354209 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-69twj" event={"ID":"93e3dae5-1990-4737-91ae-178d25c4ff1d","Type":"ContainerStarted","Data":"674caddbbd98592e1da9ba2b3d0de3c5e3487d9484c8b7b7ce3d5afef48d2454"} Oct 30 22:08:20.406697 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:20.406644 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-69twj" podStartSLOduration=1.296854915 podStartE2EDuration="2.406631967s" totalImagesPullingTime="1.109777052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:20.406223423 +0000 UTC m=+17.321909726" watchObservedRunningTime="2026-10-30 22:08:20.406631967 +0000 UTC m=+17.322318270" Oct 30 22:08:20.895868 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:20.895835 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:22.527501 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.527470 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:22.663429 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.663393 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:22.796654 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.796572 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:22.814790 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.814759 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894"] Oct 30 22:08:22.818476 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.818455 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" Oct 30 22:08:22.825576 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.825554 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fzbfr"] Oct 30 22:08:22.828834 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.828818 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:22.831260 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.831227 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Oct 30 22:08:22.831714 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.831699 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-gw6hf\"" Oct 30 22:08:22.831784 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.831750 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Oct 30 22:08:22.842777 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.842753 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894"] Oct 30 22:08:22.848912 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.848785 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cs9pw\"" Oct 30 22:08:22.848912 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.848866 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 30 22:08:22.848912 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.848893 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 30 22:08:22.880570 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.880510 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 30 22:08:22.880767 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.880724 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Oct 30 22:08:22.931587 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.931557 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:22.983185 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.983159 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-v4894\" (UID: \"b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" Oct 30 22:08:22.983347 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.983193 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d92br\" (UniqueName: \"kubernetes.io/projected/b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6-kube-api-access-d92br\") pod \"kube-state-metrics-5dc8d7d89b-v4894\" (UID: \"b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" Oct 30 22:08:22.983347 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.983220 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/829e5887-6e47-4940-8c75-8d07907ab739-root\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:22.983347 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.983320 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/829e5887-6e47-4940-8c75-8d07907ab739-metrics-client-ca\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:22.983471 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.983370 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/829e5887-6e47-4940-8c75-8d07907ab739-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:22.983471 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.983407 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-v4894\" (UID: \"b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" Oct 30 22:08:22.983471 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.983434 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-v4894\" (UID: \"b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" Oct 30 22:08:22.983568 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.983527 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/829e5887-6e47-4940-8c75-8d07907ab739-node-exporter-wtmp\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:22.983607 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.983587 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/829e5887-6e47-4940-8c75-8d07907ab739-node-exporter-tls\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:22.983652 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.983606 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/829e5887-6e47-4940-8c75-8d07907ab739-sys\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:22.983652 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.983635 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-v4894\" (UID: \"b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" Oct 30 22:08:22.983741 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.983653 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/829e5887-6e47-4940-8c75-8d07907ab739-node-exporter-accelerators-collector-config\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:22.983741 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.983696 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-v4894\" (UID: \"b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" Oct 30 22:08:22.983741 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.983728 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/829e5887-6e47-4940-8c75-8d07907ab739-node-exporter-textfile\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:22.983864 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:22.983791 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2f28\" (UniqueName: \"kubernetes.io/projected/829e5887-6e47-4940-8c75-8d07907ab739-kube-api-access-b2f28\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:23.057586 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.057509 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:23.084319 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.084287 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-v4894\" (UID: \"b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" Oct 30 22:08:23.084502 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.084333 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-v4894\" (UID: \"b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" Oct 30 22:08:23.084502 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.084364 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/829e5887-6e47-4940-8c75-8d07907ab739-node-exporter-wtmp\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:23.084502 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.084391 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/829e5887-6e47-4940-8c75-8d07907ab739-node-exporter-tls\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:23.084502 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.084416 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/829e5887-6e47-4940-8c75-8d07907ab739-sys\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:23.084502 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.084445 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-v4894\" (UID: \"b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" Oct 30 22:08:23.084767 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.084513 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/829e5887-6e47-4940-8c75-8d07907ab739-sys\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:23.084767 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.084562 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/829e5887-6e47-4940-8c75-8d07907ab739-node-exporter-wtmp\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:23.084767 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.084565 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/829e5887-6e47-4940-8c75-8d07907ab739-node-exporter-accelerators-collector-config\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:23.084767 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.084640 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-v4894\" (UID: \"b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" Oct 30 22:08:23.084767 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.084672 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/829e5887-6e47-4940-8c75-8d07907ab739-node-exporter-textfile\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:23.084767 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.084712 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2f28\" (UniqueName: \"kubernetes.io/projected/829e5887-6e47-4940-8c75-8d07907ab739-kube-api-access-b2f28\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:23.084767 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.084764 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-v4894\" (UID: \"b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" Oct 30 22:08:23.085145 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.084791 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d92br\" (UniqueName: \"kubernetes.io/projected/b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6-kube-api-access-d92br\") pod \"kube-state-metrics-5dc8d7d89b-v4894\" (UID: \"b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" Oct 30 22:08:23.085145 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.084814 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/829e5887-6e47-4940-8c75-8d07907ab739-root\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:23.085145 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.084846 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/829e5887-6e47-4940-8c75-8d07907ab739-metrics-client-ca\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:23.085145 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.084867 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-v4894\" (UID: \"b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" Oct 30 22:08:23.085337 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.085190 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-v4894\" (UID: \"b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" Oct 30 22:08:23.085337 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.085205 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/829e5887-6e47-4940-8c75-8d07907ab739-node-exporter-accelerators-collector-config\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:23.085337 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.085268 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/829e5887-6e47-4940-8c75-8d07907ab739-root\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:23.085946 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.084870 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/829e5887-6e47-4940-8c75-8d07907ab739-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:23.085946 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.085712 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/829e5887-6e47-4940-8c75-8d07907ab739-metrics-client-ca\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:23.085946 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.085809 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-v4894\" (UID: \"b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" Oct 30 22:08:23.085946 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.085894 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/829e5887-6e47-4940-8c75-8d07907ab739-node-exporter-textfile\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:23.087484 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.087438 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/829e5887-6e47-4940-8c75-8d07907ab739-node-exporter-tls\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:23.087661 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.087636 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-v4894\" (UID: \"b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" Oct 30 22:08:23.088068 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.088027 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/829e5887-6e47-4940-8c75-8d07907ab739-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:23.088397 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.088378 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-v4894\" (UID: \"b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" Oct 30 22:08:23.102955 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.102931 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d92br\" (UniqueName: \"kubernetes.io/projected/b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6-kube-api-access-d92br\") pod \"kube-state-metrics-5dc8d7d89b-v4894\" (UID: \"b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" Oct 30 22:08:23.103277 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.103257 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2f28\" (UniqueName: \"kubernetes.io/projected/829e5887-6e47-4940-8c75-8d07907ab739-kube-api-access-b2f28\") pod \"node-exporter-fzbfr\" (UID: \"829e5887-6e47-4940-8c75-8d07907ab739\") " pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:23.134778 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.134751 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" Oct 30 22:08:23.146784 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.146755 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fzbfr" Oct 30 22:08:23.156700 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:23.156673 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod829e5887_6e47_4940_8c75_8d07907ab739.slice/crio-c7af60b5a1bc5d1cf8c6a6f3bde20925df3801e925929c9ca4f4ee4d4e8d5606 WatchSource:0}: Error finding container c7af60b5a1bc5d1cf8c6a6f3bde20925df3801e925929c9ca4f4ee4d4e8d5606: Status 404 returned error can't find the container with id c7af60b5a1bc5d1cf8c6a6f3bde20925df3801e925929c9ca4f4ee4d4e8d5606 Oct 30 22:08:23.184670 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.184633 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:23.309320 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.309241 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:23.365175 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.365142 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fzbfr" event={"ID":"829e5887-6e47-4940-8c75-8d07907ab739","Type":"ContainerStarted","Data":"c7af60b5a1bc5d1cf8c6a6f3bde20925df3801e925929c9ca4f4ee4d4e8d5606"} Oct 30 22:08:23.440324 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.440294 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894"] Oct 30 22:08:23.443728 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:23.443696 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb32f8fd6_630e_46b6_98c8_cc2ac7c52cb6.slice/crio-3ca2a364588f71ba4c4004ec5fe36ecb52b50505f4353984e2ddcfbbf8b388b9 WatchSource:0}: Error finding container 3ca2a364588f71ba4c4004ec5fe36ecb52b50505f4353984e2ddcfbbf8b388b9: Status 404 returned error can't find the container with id 3ca2a364588f71ba4c4004ec5fe36ecb52b50505f4353984e2ddcfbbf8b388b9 Oct 30 22:08:23.901083 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:23.901032 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:24.016855 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.016813 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:24.138560 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.138534 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:24.155700 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.155637 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 22:08:24.160185 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.159935 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.174453 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.174428 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Oct 30 22:08:24.174575 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.174545 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Oct 30 22:08:24.174640 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.174587 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Oct 30 22:08:24.174640 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.174621 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Oct 30 22:08:24.174740 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.174642 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Oct 30 22:08:24.174740 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.174731 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-sxkkv\"" Oct 30 22:08:24.174843 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.174828 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Oct 30 22:08:24.185033 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.185006 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Oct 30 22:08:24.196063 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.196011 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Oct 30 22:08:24.196398 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.196376 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Oct 30 22:08:24.197865 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.197759 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 22:08:24.267321 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.267286 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:24.296551 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.296521 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.296681 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.296562 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19e30c8d-86ff-4018-814f-4f3515a38bfb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.296681 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.296656 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g2nm\" (UniqueName: \"kubernetes.io/projected/19e30c8d-86ff-4018-814f-4f3515a38bfb-kube-api-access-2g2nm\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.296778 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.296683 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-web-config\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.296778 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.296710 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19e30c8d-86ff-4018-814f-4f3515a38bfb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.296886 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.296858 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19e30c8d-86ff-4018-814f-4f3515a38bfb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.296950 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.296927 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.297025 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.297000 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.297109 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.297092 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-config-volume\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.297151 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.297116 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19e30c8d-86ff-4018-814f-4f3515a38bfb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.297151 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.297141 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.297236 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.297175 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19e30c8d-86ff-4018-814f-4f3515a38bfb-config-out\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.297236 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.297219 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.370597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.370564 4615 generic.go:356] "Generic (PLEG): container finished" podID="829e5887-6e47-4940-8c75-8d07907ab739" containerID="b6c84c83ba09abc4594ddca5c5d09d5cc88fb6d98c704241d4e0a3aaf15e61b5" exitCode=0 Oct 30 22:08:24.370776 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.370653 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fzbfr" event={"ID":"829e5887-6e47-4940-8c75-8d07907ab739","Type":"ContainerDied","Data":"b6c84c83ba09abc4594ddca5c5d09d5cc88fb6d98c704241d4e0a3aaf15e61b5"} Oct 30 22:08:24.371907 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.371887 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" event={"ID":"b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6","Type":"ContainerStarted","Data":"3ca2a364588f71ba4c4004ec5fe36ecb52b50505f4353984e2ddcfbbf8b388b9"} Oct 30 22:08:24.397891 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.397865 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.397993 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.397915 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.397993 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.397945 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19e30c8d-86ff-4018-814f-4f3515a38bfb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.397993 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.397978 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g2nm\" (UniqueName: \"kubernetes.io/projected/19e30c8d-86ff-4018-814f-4f3515a38bfb-kube-api-access-2g2nm\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.398167 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.398006 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-web-config\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.398167 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.398103 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19e30c8d-86ff-4018-814f-4f3515a38bfb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.398268 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.398180 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19e30c8d-86ff-4018-814f-4f3515a38bfb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.398268 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.398222 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.398268 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.398249 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.398420 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.398305 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-config-volume\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.398420 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.398328 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19e30c8d-86ff-4018-814f-4f3515a38bfb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.398420 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.398356 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.398420 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.398382 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19e30c8d-86ff-4018-814f-4f3515a38bfb-config-out\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.399078 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.398695 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19e30c8d-86ff-4018-814f-4f3515a38bfb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.400963 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.400935 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19e30c8d-86ff-4018-814f-4f3515a38bfb-config-out\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.401134 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.401112 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-config-volume\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.402343 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.402292 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19e30c8d-86ff-4018-814f-4f3515a38bfb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.402343 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.402325 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-web-config\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.402930 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.402906 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19e30c8d-86ff-4018-814f-4f3515a38bfb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.403525 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.403498 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.404066 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.404028 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.404156 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.404037 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19e30c8d-86ff-4018-814f-4f3515a38bfb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.404707 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.404685 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.405988 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.405933 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.405988 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.405952 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.415085 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.415064 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g2nm\" (UniqueName: \"kubernetes.io/projected/19e30c8d-86ff-4018-814f-4f3515a38bfb-kube-api-access-2g2nm\") pod \"alertmanager-main-0\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.498444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.498415 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:08:24.822607 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:24.822578 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 22:08:24.825808 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:24.825784 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19e30c8d_86ff_4018_814f_4f3515a38bfb.slice/crio-15a5e4d44a1ee727f17aabd3906a8dee400901aa306cceb3ee3d3c6e684f2459 WatchSource:0}: Error finding container 15a5e4d44a1ee727f17aabd3906a8dee400901aa306cceb3ee3d3c6e684f2459: Status 404 returned error can't find the container with id 15a5e4d44a1ee727f17aabd3906a8dee400901aa306cceb3ee3d3c6e684f2459 Oct 30 22:08:25.378735 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:25.378693 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fzbfr" event={"ID":"829e5887-6e47-4940-8c75-8d07907ab739","Type":"ContainerStarted","Data":"b8480ec925c176c2b75086c0f828d6bcd1079db489cb3f6ee30c062a697359a7"} Oct 30 22:08:25.379225 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:25.378759 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fzbfr" event={"ID":"829e5887-6e47-4940-8c75-8d07907ab739","Type":"ContainerStarted","Data":"881dfc6abd564721157cad9dc221f6f3185a255475250f42f99526b78dda6210"} Oct 30 22:08:25.381128 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:25.381100 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" event={"ID":"b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6","Type":"ContainerStarted","Data":"5f160c5df12a70e0983221fabfb04853e3b34a60d98e49dfd62ec51669bb73c9"} Oct 30 22:08:25.381241 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:25.381152 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" event={"ID":"b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6","Type":"ContainerStarted","Data":"30202dd04e74c1f9586f29d9e9a2442abf26076136c108c786df42724bd37178"} Oct 30 22:08:25.381241 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:25.381185 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" event={"ID":"b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6","Type":"ContainerStarted","Data":"9b311ebe63ef2f83b30a2b37bb3d4a688279f0e95fc29ebe85d3e2231e99f165"} Oct 30 22:08:25.382750 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:25.382729 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19e30c8d-86ff-4018-814f-4f3515a38bfb","Type":"ContainerStarted","Data":"15a5e4d44a1ee727f17aabd3906a8dee400901aa306cceb3ee3d3c6e684f2459"} Oct 30 22:08:25.416108 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:25.416060 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fzbfr" podStartSLOduration=2.456604438 podStartE2EDuration="3.416029508s" totalImagesPullingTime="875.4453ms" totalInitContainerRuntime="83.97977ms" isStatefulPod=true podCreationTimestamp="2026-10-30 22:08:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:25.415553189 +0000 UTC m=+22.331239503" watchObservedRunningTime="2026-10-30 22:08:25.416029508 +0000 UTC m=+22.331715812" Oct 30 22:08:25.452187 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:25.452130 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-v4894" podStartSLOduration=2.118669143 podStartE2EDuration="3.452112399s" totalImagesPullingTime="1.333443256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:25.451346219 +0000 UTC m=+22.367032524" watchObservedRunningTime="2026-10-30 22:08:25.452112399 +0000 UTC m=+22.367798716" Oct 30 22:08:26.057916 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:26.057837 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:26.178509 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:26.178475 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:26.295420 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:26.295387 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:26.387845 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:26.387814 4615 generic.go:356] "Generic (PLEG): container finished" podID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerID="39eb90131d7672ec1b9b06c56fd0baba6f178647349316af9c6b710563b6586f" exitCode=0 Oct 30 22:08:26.388262 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:26.387904 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19e30c8d-86ff-4018-814f-4f3515a38bfb","Type":"ContainerDied","Data":"39eb90131d7672ec1b9b06c56fd0baba6f178647349316af9c6b710563b6586f"} Oct 30 22:08:26.481107 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:26.481073 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:26.596659 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:26.596631 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:27.258179 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.258143 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-64c486767d-86tlj"] Oct 30 22:08:27.263389 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.263370 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.291769 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.291739 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Oct 30 22:08:27.291933 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.291803 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Oct 30 22:08:27.292005 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.291933 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-50pvtoch19mrk\"" Oct 30 22:08:27.292005 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.291999 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qsncb\"" Oct 30 22:08:27.292210 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.292068 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Oct 30 22:08:27.292210 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.292136 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Oct 30 22:08:27.292875 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.292849 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-64c486767d-86tlj"] Oct 30 22:08:27.424639 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.424606 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1fcba11f-af23-40ad-8f18-f3a14c48d1d5-secret-metrics-server-client-certs\") pod \"metrics-server-64c486767d-86tlj\" (UID: \"1fcba11f-af23-40ad-8f18-f3a14c48d1d5\") " pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.424639 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.424646 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fcba11f-af23-40ad-8f18-f3a14c48d1d5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-64c486767d-86tlj\" (UID: \"1fcba11f-af23-40ad-8f18-f3a14c48d1d5\") " pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.425146 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.424768 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5xg5\" (UniqueName: \"kubernetes.io/projected/1fcba11f-af23-40ad-8f18-f3a14c48d1d5-kube-api-access-p5xg5\") pod \"metrics-server-64c486767d-86tlj\" (UID: \"1fcba11f-af23-40ad-8f18-f3a14c48d1d5\") " pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.425146 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.424828 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1fcba11f-af23-40ad-8f18-f3a14c48d1d5-audit-log\") pod \"metrics-server-64c486767d-86tlj\" (UID: \"1fcba11f-af23-40ad-8f18-f3a14c48d1d5\") " pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.425146 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.424875 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1fcba11f-af23-40ad-8f18-f3a14c48d1d5-client-ca-bundle\") pod \"metrics-server-64c486767d-86tlj\" (UID: \"1fcba11f-af23-40ad-8f18-f3a14c48d1d5\") " pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.425146 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.424995 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1fcba11f-af23-40ad-8f18-f3a14c48d1d5-metrics-server-audit-profiles\") pod \"metrics-server-64c486767d-86tlj\" (UID: \"1fcba11f-af23-40ad-8f18-f3a14c48d1d5\") " pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.425146 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.425024 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1fcba11f-af23-40ad-8f18-f3a14c48d1d5-secret-metrics-server-tls\") pod \"metrics-server-64c486767d-86tlj\" (UID: \"1fcba11f-af23-40ad-8f18-f3a14c48d1d5\") " pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.527461 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.527373 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1fcba11f-af23-40ad-8f18-f3a14c48d1d5-client-ca-bundle\") pod \"metrics-server-64c486767d-86tlj\" (UID: \"1fcba11f-af23-40ad-8f18-f3a14c48d1d5\") " pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.527461 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.527428 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1fcba11f-af23-40ad-8f18-f3a14c48d1d5-metrics-server-audit-profiles\") pod \"metrics-server-64c486767d-86tlj\" (UID: \"1fcba11f-af23-40ad-8f18-f3a14c48d1d5\") " pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.528073 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.527900 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1fcba11f-af23-40ad-8f18-f3a14c48d1d5-secret-metrics-server-tls\") pod \"metrics-server-64c486767d-86tlj\" (UID: \"1fcba11f-af23-40ad-8f18-f3a14c48d1d5\") " pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.528073 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.527949 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1fcba11f-af23-40ad-8f18-f3a14c48d1d5-secret-metrics-server-client-certs\") pod \"metrics-server-64c486767d-86tlj\" (UID: \"1fcba11f-af23-40ad-8f18-f3a14c48d1d5\") " pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.528073 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.527981 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fcba11f-af23-40ad-8f18-f3a14c48d1d5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-64c486767d-86tlj\" (UID: \"1fcba11f-af23-40ad-8f18-f3a14c48d1d5\") " pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.528073 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.528027 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5xg5\" (UniqueName: \"kubernetes.io/projected/1fcba11f-af23-40ad-8f18-f3a14c48d1d5-kube-api-access-p5xg5\") pod \"metrics-server-64c486767d-86tlj\" (UID: \"1fcba11f-af23-40ad-8f18-f3a14c48d1d5\") " pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.528826 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.528438 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1fcba11f-af23-40ad-8f18-f3a14c48d1d5-audit-log\") pod \"metrics-server-64c486767d-86tlj\" (UID: \"1fcba11f-af23-40ad-8f18-f3a14c48d1d5\") " pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.528826 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.528618 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1fcba11f-af23-40ad-8f18-f3a14c48d1d5-metrics-server-audit-profiles\") pod \"metrics-server-64c486767d-86tlj\" (UID: \"1fcba11f-af23-40ad-8f18-f3a14c48d1d5\") " pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.528826 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.528740 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1fcba11f-af23-40ad-8f18-f3a14c48d1d5-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-64c486767d-86tlj\" (UID: \"1fcba11f-af23-40ad-8f18-f3a14c48d1d5\") " pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.528826 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.528769 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1fcba11f-af23-40ad-8f18-f3a14c48d1d5-audit-log\") pod \"metrics-server-64c486767d-86tlj\" (UID: \"1fcba11f-af23-40ad-8f18-f3a14c48d1d5\") " pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.530431 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.530408 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1fcba11f-af23-40ad-8f18-f3a14c48d1d5-client-ca-bundle\") pod \"metrics-server-64c486767d-86tlj\" (UID: \"1fcba11f-af23-40ad-8f18-f3a14c48d1d5\") " pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.530705 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.530683 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1fcba11f-af23-40ad-8f18-f3a14c48d1d5-secret-metrics-server-tls\") pod \"metrics-server-64c486767d-86tlj\" (UID: \"1fcba11f-af23-40ad-8f18-f3a14c48d1d5\") " pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.530785 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.530721 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1fcba11f-af23-40ad-8f18-f3a14c48d1d5-secret-metrics-server-client-certs\") pod \"metrics-server-64c486767d-86tlj\" (UID: \"1fcba11f-af23-40ad-8f18-f3a14c48d1d5\") " pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.549671 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.549642 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5xg5\" (UniqueName: \"kubernetes.io/projected/1fcba11f-af23-40ad-8f18-f3a14c48d1d5-kube-api-access-p5xg5\") pod \"metrics-server-64c486767d-86tlj\" (UID: \"1fcba11f-af23-40ad-8f18-f3a14c48d1d5\") " pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.577987 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.577957 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:27.627410 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.627349 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:27.734682 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.734655 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:27.921357 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:27.921333 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-64c486767d-86tlj"] Oct 30 22:08:27.922418 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:27.922393 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fcba11f_af23_40ad_8f18_f3a14c48d1d5.slice/crio-b9500e6ce4b563f846b839881dbde49acc1c2a5f9cd4ed7296ea7291f9b16a18 WatchSource:0}: Error finding container b9500e6ce4b563f846b839881dbde49acc1c2a5f9cd4ed7296ea7291f9b16a18: Status 404 returned error can't find the container with id b9500e6ce4b563f846b839881dbde49acc1c2a5f9cd4ed7296ea7291f9b16a18 Oct 30 22:08:28.395963 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:28.395936 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-64c486767d-86tlj" event={"ID":"1fcba11f-af23-40ad-8f18-f3a14c48d1d5","Type":"ContainerStarted","Data":"b9500e6ce4b563f846b839881dbde49acc1c2a5f9cd4ed7296ea7291f9b16a18"} Oct 30 22:08:28.398834 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:28.398807 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19e30c8d-86ff-4018-814f-4f3515a38bfb","Type":"ContainerStarted","Data":"bfd52530fc208adc75492c1da99fdd3065ea83539cecec018e88d94a48c45891"} Oct 30 22:08:28.398918 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:28.398876 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19e30c8d-86ff-4018-814f-4f3515a38bfb","Type":"ContainerStarted","Data":"e645ec4cfd933857bb7a7b4ef26f90d9fa1d6f6c8b5183e9ed05ce893e65fa26"} Oct 30 22:08:28.398957 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:28.398920 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19e30c8d-86ff-4018-814f-4f3515a38bfb","Type":"ContainerStarted","Data":"b0110e357a28e74b1477befec5370e6b7636e2f0b6e9c1d63a104db2f3d63bc6"} Oct 30 22:08:29.405272 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:29.405237 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19e30c8d-86ff-4018-814f-4f3515a38bfb","Type":"ContainerStarted","Data":"b8d9847a7548ad3c9f2ae8cf1e38f9628613deddd40b92f1e13f8082626ea079"} Oct 30 22:08:29.405681 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:29.405301 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19e30c8d-86ff-4018-814f-4f3515a38bfb","Type":"ContainerStarted","Data":"e4f41d220d3530a3d37ab2826d51fc120735e57c2e09ac09678c71c8fefc24db"} Oct 30 22:08:29.728943 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:29.728825 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:29.770641 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:08:29.770601 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:08:30.412751 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:30.412712 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19e30c8d-86ff-4018-814f-4f3515a38bfb","Type":"ContainerStarted","Data":"adacc828579c0c2194152ec747945ea2782bbb58afe5372ddc5cb7a73167f069"} Oct 30 22:08:30.414181 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:30.414152 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-64c486767d-86tlj" event={"ID":"1fcba11f-af23-40ad-8f18-f3a14c48d1d5","Type":"ContainerStarted","Data":"f7c79a41e0e2f2f510c4e5d9509247b59e74132d3ebe137207285461d6c89af4"} Oct 30 22:08:30.489496 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:30.489428 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.380196289 podStartE2EDuration="6.489407961s" totalImagesPullingTime="4.096658306s" totalInitContainerRuntime="12.553366ms" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:30.488645716 +0000 UTC m=+27.404332051" watchObservedRunningTime="2026-10-30 22:08:30.489407961 +0000 UTC m=+27.405094264" Oct 30 22:08:30.530694 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:30.530633 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-64c486767d-86tlj" podStartSLOduration=1.520410251 podStartE2EDuration="3.530612393s" totalImagesPullingTime="2.010202142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:30.529510974 +0000 UTC m=+27.445197280" watchObservedRunningTime="2026-10-30 22:08:30.530612393 +0000 UTC m=+27.446298697" Oct 30 22:08:30.592969 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:30.592937 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wqj4x" Oct 30 22:08:30.647399 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:30.647365 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:30.791640 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:30.791537 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:32.180722 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:32.180667 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:32.294616 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:32.294582 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:35.624569 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:35.624535 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:35.754648 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:35.754614 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:35.869019 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:35.868983 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:35.988799 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:35.988768 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:36.216559 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:36.216527 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-cpcvr"] Oct 30 22:08:36.250322 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:36.250243 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-cpcvr"] Oct 30 22:08:36.250472 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:36.250377 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-cpcvr" Oct 30 22:08:36.265817 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:36.265790 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Oct 30 22:08:36.265817 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:36.265797 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Oct 30 22:08:36.309638 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:36.309609 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12607685-74bf-471b-9c1d-ae94e6e9c6c7-tmp\") pod \"downloads-6dd789c8b7-cpcvr\" (UID: \"12607685-74bf-471b-9c1d-ae94e6e9c6c7\") " pod="openshift-console/downloads-6dd789c8b7-cpcvr" Oct 30 22:08:36.309815 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:36.309659 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsj25\" (UniqueName: \"kubernetes.io/projected/12607685-74bf-471b-9c1d-ae94e6e9c6c7-kube-api-access-dsj25\") pod \"downloads-6dd789c8b7-cpcvr\" (UID: \"12607685-74bf-471b-9c1d-ae94e6e9c6c7\") " pod="openshift-console/downloads-6dd789c8b7-cpcvr" Oct 30 22:08:36.327450 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:36.327427 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-xqkfc\"" Oct 30 22:08:36.410412 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:36.410373 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsj25\" (UniqueName: \"kubernetes.io/projected/12607685-74bf-471b-9c1d-ae94e6e9c6c7-kube-api-access-dsj25\") pod \"downloads-6dd789c8b7-cpcvr\" (UID: \"12607685-74bf-471b-9c1d-ae94e6e9c6c7\") " pod="openshift-console/downloads-6dd789c8b7-cpcvr" Oct 30 22:08:36.410644 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:36.410470 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12607685-74bf-471b-9c1d-ae94e6e9c6c7-tmp\") pod \"downloads-6dd789c8b7-cpcvr\" (UID: \"12607685-74bf-471b-9c1d-ae94e6e9c6c7\") " pod="openshift-console/downloads-6dd789c8b7-cpcvr" Oct 30 22:08:36.411025 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:36.410869 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12607685-74bf-471b-9c1d-ae94e6e9c6c7-tmp\") pod \"downloads-6dd789c8b7-cpcvr\" (UID: \"12607685-74bf-471b-9c1d-ae94e6e9c6c7\") " pod="openshift-console/downloads-6dd789c8b7-cpcvr" Oct 30 22:08:36.432144 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:36.432110 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsj25\" (UniqueName: \"kubernetes.io/projected/12607685-74bf-471b-9c1d-ae94e6e9c6c7-kube-api-access-dsj25\") pod \"downloads-6dd789c8b7-cpcvr\" (UID: \"12607685-74bf-471b-9c1d-ae94e6e9c6c7\") " pod="openshift-console/downloads-6dd789c8b7-cpcvr" Oct 30 22:08:36.481975 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:36.481939 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:36.572106 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:36.572012 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-cpcvr" Oct 30 22:08:36.885534 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:36.885502 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-cpcvr"] Oct 30 22:08:36.898848 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:36.898818 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12607685_74bf_471b_9c1d_ae94e6e9c6c7.slice/crio-65e934229195340f97a75899b5b437eb93b8978d0262295c531c0cb4e68dc4d9 WatchSource:0}: Error finding container 65e934229195340f97a75899b5b437eb93b8978d0262295c531c0cb4e68dc4d9: Status 404 returned error can't find the container with id 65e934229195340f97a75899b5b437eb93b8978d0262295c531c0cb4e68dc4d9 Oct 30 22:08:37.079736 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:37.079699 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:37.197453 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:37.197365 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:37.315366 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:37.315333 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:37.444665 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:37.444627 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-cpcvr" event={"ID":"12607685-74bf-471b-9c1d-ae94e6e9c6c7","Type":"ContainerStarted","Data":"65e934229195340f97a75899b5b437eb93b8978d0262295c531c0cb4e68dc4d9"} Oct 30 22:08:37.554003 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:37.553925 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:37.670985 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:37.670942 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:38.253261 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:38.253227 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:38.415549 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:38.415509 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:38.872393 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:38.872355 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pmkb2" Oct 30 22:08:41.486297 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.486262 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:41.549444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.549399 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-9f78dd5d-nnrh8"] Oct 30 22:08:41.551903 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.551882 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:41.567456 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.567431 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Oct 30 22:08:41.581072 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.579895 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Oct 30 22:08:41.581072 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.579921 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Oct 30 22:08:41.581430 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.581406 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9f78dd5d-nnrh8"] Oct 30 22:08:41.588934 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.588788 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Oct 30 22:08:41.589144 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.588816 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vp8lb\"" Oct 30 22:08:41.589144 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.588853 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Oct 30 22:08:41.612001 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.611968 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:41.661337 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.661305 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef992b4b-7679-41b3-bfed-ea21a49be22a-console-serving-cert\") pod \"console-9f78dd5d-nnrh8\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:41.661526 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.661347 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef992b4b-7679-41b3-bfed-ea21a49be22a-oauth-serving-cert\") pod \"console-9f78dd5d-nnrh8\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:41.661526 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.661439 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-955b9\" (UniqueName: \"kubernetes.io/projected/ef992b4b-7679-41b3-bfed-ea21a49be22a-kube-api-access-955b9\") pod \"console-9f78dd5d-nnrh8\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:41.661526 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.661488 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef992b4b-7679-41b3-bfed-ea21a49be22a-console-config\") pod \"console-9f78dd5d-nnrh8\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:41.661526 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.661517 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef992b4b-7679-41b3-bfed-ea21a49be22a-service-ca\") pod \"console-9f78dd5d-nnrh8\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:41.661693 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.661547 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef992b4b-7679-41b3-bfed-ea21a49be22a-console-oauth-config\") pod \"console-9f78dd5d-nnrh8\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:41.661693 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.661670 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef992b4b-7679-41b3-bfed-ea21a49be22a-tmp\") pod \"console-9f78dd5d-nnrh8\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:41.762787 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.762712 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef992b4b-7679-41b3-bfed-ea21a49be22a-oauth-serving-cert\") pod \"console-9f78dd5d-nnrh8\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:41.762787 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.762772 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-955b9\" (UniqueName: \"kubernetes.io/projected/ef992b4b-7679-41b3-bfed-ea21a49be22a-kube-api-access-955b9\") pod \"console-9f78dd5d-nnrh8\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:41.763018 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.762799 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef992b4b-7679-41b3-bfed-ea21a49be22a-console-config\") pod \"console-9f78dd5d-nnrh8\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:41.763018 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.762823 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef992b4b-7679-41b3-bfed-ea21a49be22a-service-ca\") pod \"console-9f78dd5d-nnrh8\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:41.763018 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.762856 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef992b4b-7679-41b3-bfed-ea21a49be22a-console-oauth-config\") pod \"console-9f78dd5d-nnrh8\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:41.763237 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.763217 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef992b4b-7679-41b3-bfed-ea21a49be22a-tmp\") pod \"console-9f78dd5d-nnrh8\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:41.763299 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.763264 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef992b4b-7679-41b3-bfed-ea21a49be22a-console-serving-cert\") pod \"console-9f78dd5d-nnrh8\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:41.763642 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.763610 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef992b4b-7679-41b3-bfed-ea21a49be22a-oauth-serving-cert\") pod \"console-9f78dd5d-nnrh8\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:41.763785 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.763639 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef992b4b-7679-41b3-bfed-ea21a49be22a-console-config\") pod \"console-9f78dd5d-nnrh8\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:41.763785 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.763758 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef992b4b-7679-41b3-bfed-ea21a49be22a-tmp\") pod \"console-9f78dd5d-nnrh8\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:41.763917 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.763856 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef992b4b-7679-41b3-bfed-ea21a49be22a-service-ca\") pod \"console-9f78dd5d-nnrh8\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:41.765636 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.765613 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef992b4b-7679-41b3-bfed-ea21a49be22a-console-oauth-config\") pod \"console-9f78dd5d-nnrh8\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:41.765843 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.765822 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef992b4b-7679-41b3-bfed-ea21a49be22a-console-serving-cert\") pod \"console-9f78dd5d-nnrh8\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:41.784005 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.783977 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-955b9\" (UniqueName: \"kubernetes.io/projected/ef992b4b-7679-41b3-bfed-ea21a49be22a-kube-api-access-955b9\") pod \"console-9f78dd5d-nnrh8\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:41.864306 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:41.864258 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:42.168712 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:42.168688 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9f78dd5d-nnrh8"] Oct 30 22:08:42.169937 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:42.169899 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef992b4b_7679_41b3_bfed_ea21a49be22a.slice/crio-e9578b4988bed5dbe79565300b830fee7314e951ec7d7b6e0cc6454e2b825b5f WatchSource:0}: Error finding container e9578b4988bed5dbe79565300b830fee7314e951ec7d7b6e0cc6454e2b825b5f: Status 404 returned error can't find the container with id e9578b4988bed5dbe79565300b830fee7314e951ec7d7b6e0cc6454e2b825b5f Oct 30 22:08:42.466922 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:42.466843 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9f78dd5d-nnrh8" event={"ID":"ef992b4b-7679-41b3-bfed-ea21a49be22a","Type":"ContainerStarted","Data":"e9578b4988bed5dbe79565300b830fee7314e951ec7d7b6e0cc6454e2b825b5f"} Oct 30 22:08:42.656786 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:42.656738 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:42.781638 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:42.781562 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:43.160350 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:43.160311 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:45.486906 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:45.486864 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9f78dd5d-nnrh8" event={"ID":"ef992b4b-7679-41b3-bfed-ea21a49be22a","Type":"ContainerStarted","Data":"aea5dbec327495dcaa879e7f8b2b7ef79fc93133d6bb520648b9dd87243fe98c"} Oct 30 22:08:45.528698 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:45.528635 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-9f78dd5d-nnrh8" podStartSLOduration=1.576852989 podStartE2EDuration="4.528616515s" totalImagesPullingTime="2.951763526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:45.528295946 +0000 UTC m=+42.443982250" watchObservedRunningTime="2026-10-30 22:08:45.528616515 +0000 UTC m=+42.444302819" Oct 30 22:08:47.578417 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:47.578185 4615 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:47.578417 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:47.578271 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:08:48.321272 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:48.321228 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:48.458729 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:48.458692 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:51.865412 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:51.865374 4615 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:51.865906 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:51.865489 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:51.870792 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:51.870761 4615 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:52.517406 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:52.517368 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-cpcvr" event={"ID":"12607685-74bf-471b-9c1d-ae94e6e9c6c7","Type":"ContainerStarted","Data":"e9cf27af43d86e1c289291ac5aa26588e75e9a81e932578b23ebd2250aa98ca6"} Oct 30 22:08:52.521197 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:52.521168 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:08:52.558972 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:52.558939 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:52.632671 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:52.631338 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-cpcvr" podStartSLOduration=1.151036368 podStartE2EDuration="16.6313168s" totalImagesPullingTime="15.480280432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:52.589204724 +0000 UTC m=+49.504891027" watchObservedRunningTime="2026-10-30 22:08:52.6313168 +0000 UTC m=+49.547003103" Oct 30 22:08:52.683730 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:52.683676 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:53.338454 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.338419 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-68c99c9bdd-h582d"] Oct 30 22:08:53.345135 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.345106 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.399219 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.399183 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Oct 30 22:08:53.400611 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.400583 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68c99c9bdd-h582d"] Oct 30 22:08:53.471521 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.471472 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-console-config\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.471723 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.471595 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-trusted-ca-bundle\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.471723 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.471637 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cf0c1e02-8817-4c08-a9c1-38fb654e6761-console-serving-cert\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.471864 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.471734 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cf0c1e02-8817-4c08-a9c1-38fb654e6761-console-oauth-config\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.471864 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.471757 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-service-ca\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.471864 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.471836 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf0c1e02-8817-4c08-a9c1-38fb654e6761-tmp\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.472023 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.471914 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hb68\" (UniqueName: \"kubernetes.io/projected/cf0c1e02-8817-4c08-a9c1-38fb654e6761-kube-api-access-5hb68\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.472023 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.471955 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-oauth-serving-cert\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.522126 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.522093 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-cpcvr" Oct 30 22:08:53.541152 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.541120 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-cpcvr" Oct 30 22:08:53.572628 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.572588 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-trusted-ca-bundle\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.572628 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.572628 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cf0c1e02-8817-4c08-a9c1-38fb654e6761-console-serving-cert\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.572886 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.572664 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cf0c1e02-8817-4c08-a9c1-38fb654e6761-console-oauth-config\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.572886 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.572688 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-service-ca\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.572886 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.572732 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf0c1e02-8817-4c08-a9c1-38fb654e6761-tmp\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.572886 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.572786 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hb68\" (UniqueName: \"kubernetes.io/projected/cf0c1e02-8817-4c08-a9c1-38fb654e6761-kube-api-access-5hb68\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.572886 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.572828 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-oauth-serving-cert\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.573169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.572879 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-console-config\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.573968 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.573533 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf0c1e02-8817-4c08-a9c1-38fb654e6761-tmp\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.573968 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.573779 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-console-config\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.574225 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.574203 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-service-ca\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.574289 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.574250 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-oauth-serving-cert\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.574822 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.574794 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-trusted-ca-bundle\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.575705 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.575686 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cf0c1e02-8817-4c08-a9c1-38fb654e6761-console-serving-cert\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.575816 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.575734 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cf0c1e02-8817-4c08-a9c1-38fb654e6761-console-oauth-config\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.594904 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.594839 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hb68\" (UniqueName: \"kubernetes.io/projected/cf0c1e02-8817-4c08-a9c1-38fb654e6761-kube-api-access-5hb68\") pod \"console-68c99c9bdd-h582d\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.660481 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.660444 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:08:53.974359 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:53.974300 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68c99c9bdd-h582d"] Oct 30 22:08:53.976828 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:08:53.976797 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf0c1e02_8817_4c08_a9c1_38fb654e6761.slice/crio-7a8e29ba1faaf5cadd9918397c51eb0ff3ee01e445c63936c7581bb01b9f2352 WatchSource:0}: Error finding container 7a8e29ba1faaf5cadd9918397c51eb0ff3ee01e445c63936c7581bb01b9f2352: Status 404 returned error can't find the container with id 7a8e29ba1faaf5cadd9918397c51eb0ff3ee01e445c63936c7581bb01b9f2352 Oct 30 22:08:54.528711 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:54.528664 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68c99c9bdd-h582d" event={"ID":"cf0c1e02-8817-4c08-a9c1-38fb654e6761","Type":"ContainerStarted","Data":"71839e2963698832bbbb37a5eb2e2523a2c75b641ca6511a44ef748705cebbf5"} Oct 30 22:08:54.529257 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:54.528769 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68c99c9bdd-h582d" event={"ID":"cf0c1e02-8817-4c08-a9c1-38fb654e6761","Type":"ContainerStarted","Data":"7a8e29ba1faaf5cadd9918397c51eb0ff3ee01e445c63936c7581bb01b9f2352"} Oct 30 22:08:56.485157 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:56.485116 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:56.608059 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:56.608006 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:57.558147 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:57.558111 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:57.674329 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:08:57.674289 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:08:59.768643 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:08:59.768594 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:09:03.660740 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:03.660696 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:09:03.661355 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:03.660781 4615 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:09:03.665660 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:03.665639 4615 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:09:03.708611 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:03.708550 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-68c99c9bdd-h582d" podStartSLOduration=10.708537731 podStartE2EDuration="10.708537731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:08:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:08:54.571246425 +0000 UTC m=+51.486932732" watchObservedRunningTime="2026-10-30 22:09:03.708537731 +0000 UTC m=+60.624224103" Oct 30 22:09:04.571890 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:04.571856 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:09:04.659924 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:04.659888 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9f78dd5d-nnrh8"] Oct 30 22:09:04.964769 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:04.964737 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:09:05.083593 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:05.083554 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:09:06.953526 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:06.953469 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:09:07.063690 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:07.063659 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:09:07.427922 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:07.427878 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:09:07.542104 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:07.542073 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:09:07.583397 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:07.583372 4615 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:09:07.586962 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:07.586945 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-64c486767d-86tlj" Oct 30 22:09:07.649489 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:07.649460 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:09:10.353640 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:10.353605 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:09:10.468803 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:10.468769 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:09:19.712931 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:19.712897 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-r9szz-8wv8c"] Oct 30 22:09:19.713504 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:19.713130 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-r9szz-8wv8c" podUID="04f40ed9-0eef-4946-b3e6-feafcfee8e03" containerName="insights-gathering" containerID="cri-o://3c39f13c95a43f2169a67c66d6271cf08a4a4bd9e2cffe02b69c6d5e7d359a8e" gracePeriod=30 Oct 30 22:09:20.006357 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:20.006326 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:09:20.104036 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:20.104000 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04f40ed9-0eef-4946-b3e6-feafcfee8e03-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/04f40ed9-0eef-4946-b3e6-feafcfee8e03-archives-path\") pod \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " Oct 30 22:09:20.104231 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:20.104109 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/04f40ed9-0eef-4946-b3e6-feafcfee8e03-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04f40ed9-0eef-4946-b3e6-feafcfee8e03-service-ca-bundle\") pod \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " Oct 30 22:09:20.104231 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:20.104141 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04f40ed9-0eef-4946-b3e6-feafcfee8e03-kube-api-access-szj9w\" (UniqueName: \"kubernetes.io/projected/04f40ed9-0eef-4946-b3e6-feafcfee8e03-kube-api-access-szj9w\") pod \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " Oct 30 22:09:20.104231 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:20.104187 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/04f40ed9-0eef-4946-b3e6-feafcfee8e03-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04f40ed9-0eef-4946-b3e6-feafcfee8e03-trusted-ca-bundle\") pod \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\" (UID: \"04f40ed9-0eef-4946-b3e6-feafcfee8e03\") " Oct 30 22:09:20.104439 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:20.104414 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04f40ed9-0eef-4946-b3e6-feafcfee8e03-archives-path" pod "04f40ed9-0eef-4946-b3e6-feafcfee8e03" (UID: "04f40ed9-0eef-4946-b3e6-feafcfee8e03"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:09:20.104510 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:20.104450 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04f40ed9-0eef-4946-b3e6-feafcfee8e03-service-ca-bundle" pod "04f40ed9-0eef-4946-b3e6-feafcfee8e03" (UID: "04f40ed9-0eef-4946-b3e6-feafcfee8e03"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:09:20.104671 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:20.104649 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04f40ed9-0eef-4946-b3e6-feafcfee8e03-trusted-ca-bundle" pod "04f40ed9-0eef-4946-b3e6-feafcfee8e03" (UID: "04f40ed9-0eef-4946-b3e6-feafcfee8e03"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:09:20.106391 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:20.106367 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04f40ed9-0eef-4946-b3e6-feafcfee8e03-kube-api-access-szj9w" pod "04f40ed9-0eef-4946-b3e6-feafcfee8e03" (UID: "04f40ed9-0eef-4946-b3e6-feafcfee8e03"). InnerVolumeSpecName "kube-api-access-szj9w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:09:20.205718 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:20.205680 4615 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/04f40ed9-0eef-4946-b3e6-feafcfee8e03-archives-path\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:20.205718 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:20.205712 4615 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04f40ed9-0eef-4946-b3e6-feafcfee8e03-service-ca-bundle\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:20.205718 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:20.205722 4615 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szj9w\" (UniqueName: \"kubernetes.io/projected/04f40ed9-0eef-4946-b3e6-feafcfee8e03-kube-api-access-szj9w\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:20.205988 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:20.205733 4615 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04f40ed9-0eef-4946-b3e6-feafcfee8e03-trusted-ca-bundle\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:20.625461 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:20.625425 4615 generic.go:356] "Generic (PLEG): container finished" podID="04f40ed9-0eef-4946-b3e6-feafcfee8e03" containerID="3c39f13c95a43f2169a67c66d6271cf08a4a4bd9e2cffe02b69c6d5e7d359a8e" exitCode=2 Oct 30 22:09:20.625636 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:20.625473 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-r9szz-8wv8c" Oct 30 22:09:20.625636 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:20.625516 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-r9szz-8wv8c" event={"ID":"04f40ed9-0eef-4946-b3e6-feafcfee8e03","Type":"ContainerDied","Data":"3c39f13c95a43f2169a67c66d6271cf08a4a4bd9e2cffe02b69c6d5e7d359a8e"} Oct 30 22:09:20.625636 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:20.625570 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-r9szz-8wv8c" event={"ID":"04f40ed9-0eef-4946-b3e6-feafcfee8e03","Type":"ContainerDied","Data":"599f89938d5df802596ebac1c4370ccd7550633902cc5876c877b1193910f301"} Oct 30 22:09:20.625636 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:20.625605 4615 scope.go:122] "RemoveContainer" containerID="3c39f13c95a43f2169a67c66d6271cf08a4a4bd9e2cffe02b69c6d5e7d359a8e" Oct 30 22:09:20.637638 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:20.637615 4615 scope.go:122] "RemoveContainer" containerID="3c39f13c95a43f2169a67c66d6271cf08a4a4bd9e2cffe02b69c6d5e7d359a8e" Oct 30 22:09:20.637965 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:09:20.637931 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c39f13c95a43f2169a67c66d6271cf08a4a4bd9e2cffe02b69c6d5e7d359a8e\": container with ID starting with 3c39f13c95a43f2169a67c66d6271cf08a4a4bd9e2cffe02b69c6d5e7d359a8e not found: ID does not exist" containerID="3c39f13c95a43f2169a67c66d6271cf08a4a4bd9e2cffe02b69c6d5e7d359a8e" Oct 30 22:09:20.638014 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:20.637975 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c39f13c95a43f2169a67c66d6271cf08a4a4bd9e2cffe02b69c6d5e7d359a8e"} err="failed to get container status \"3c39f13c95a43f2169a67c66d6271cf08a4a4bd9e2cffe02b69c6d5e7d359a8e\": rpc error: code = NotFound desc = could not find container \"3c39f13c95a43f2169a67c66d6271cf08a4a4bd9e2cffe02b69c6d5e7d359a8e\": container with ID starting with 3c39f13c95a43f2169a67c66d6271cf08a4a4bd9e2cffe02b69c6d5e7d359a8e not found: ID does not exist" Oct 30 22:09:20.676644 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:20.676601 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-r9szz-8wv8c"] Oct 30 22:09:20.688936 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:20.688902 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-r9szz-8wv8c"] Oct 30 22:09:21.241103 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:21.241063 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="04f40ed9-0eef-4946-b3e6-feafcfee8e03" path="/var/lib/kubelet/pods/04f40ed9-0eef-4946-b3e6-feafcfee8e03/volumes" Oct 30 22:09:26.487182 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:26.487147 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:09:26.595975 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:26.595943 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:09:29.521237 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:29.521198 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:09:29.680364 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:29.680323 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-9f78dd5d-nnrh8" podUID="ef992b4b-7679-41b3-bfed-ea21a49be22a" containerName="console" containerID="cri-o://aea5dbec327495dcaa879e7f8b2b7ef79fc93133d6bb520648b9dd87243fe98c" gracePeriod=15 Oct 30 22:09:29.764512 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:09:29.764473 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:09:29.977379 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:29.977342 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:09:30.089034 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.088948 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef992b4b-7679-41b3-bfed-ea21a49be22a-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef992b4b-7679-41b3-bfed-ea21a49be22a-console-oauth-config\") pod \"ef992b4b-7679-41b3-bfed-ea21a49be22a\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " Oct 30 22:09:30.089034 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.088993 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef992b4b-7679-41b3-bfed-ea21a49be22a-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef992b4b-7679-41b3-bfed-ea21a49be22a-service-ca\") pod \"ef992b4b-7679-41b3-bfed-ea21a49be22a\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " Oct 30 22:09:30.089034 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.089040 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef992b4b-7679-41b3-bfed-ea21a49be22a-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef992b4b-7679-41b3-bfed-ea21a49be22a-console-serving-cert\") pod \"ef992b4b-7679-41b3-bfed-ea21a49be22a\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " Oct 30 22:09:30.089397 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.089106 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef992b4b-7679-41b3-bfed-ea21a49be22a-kube-api-access-955b9\" (UniqueName: \"kubernetes.io/projected/ef992b4b-7679-41b3-bfed-ea21a49be22a-kube-api-access-955b9\") pod \"ef992b4b-7679-41b3-bfed-ea21a49be22a\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " Oct 30 22:09:30.089397 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.089132 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef992b4b-7679-41b3-bfed-ea21a49be22a-tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef992b4b-7679-41b3-bfed-ea21a49be22a-tmp\") pod \"ef992b4b-7679-41b3-bfed-ea21a49be22a\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " Oct 30 22:09:30.089397 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.089157 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef992b4b-7679-41b3-bfed-ea21a49be22a-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef992b4b-7679-41b3-bfed-ea21a49be22a-oauth-serving-cert\") pod \"ef992b4b-7679-41b3-bfed-ea21a49be22a\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " Oct 30 22:09:30.089397 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.089196 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef992b4b-7679-41b3-bfed-ea21a49be22a-console-config\" (UniqueName: \"kubernetes.io/configmap/ef992b4b-7679-41b3-bfed-ea21a49be22a-console-config\") pod \"ef992b4b-7679-41b3-bfed-ea21a49be22a\" (UID: \"ef992b4b-7679-41b3-bfed-ea21a49be22a\") " Oct 30 22:09:30.089599 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.089419 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef992b4b-7679-41b3-bfed-ea21a49be22a-tmp" pod "ef992b4b-7679-41b3-bfed-ea21a49be22a" (UID: "ef992b4b-7679-41b3-bfed-ea21a49be22a"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:09:30.089599 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.089548 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef992b4b-7679-41b3-bfed-ea21a49be22a-service-ca" pod "ef992b4b-7679-41b3-bfed-ea21a49be22a" (UID: "ef992b4b-7679-41b3-bfed-ea21a49be22a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:09:30.089710 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.089651 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef992b4b-7679-41b3-bfed-ea21a49be22a-console-config" pod "ef992b4b-7679-41b3-bfed-ea21a49be22a" (UID: "ef992b4b-7679-41b3-bfed-ea21a49be22a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:09:30.089756 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.089717 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef992b4b-7679-41b3-bfed-ea21a49be22a-oauth-serving-cert" pod "ef992b4b-7679-41b3-bfed-ea21a49be22a" (UID: "ef992b4b-7679-41b3-bfed-ea21a49be22a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:09:30.091400 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.091376 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef992b4b-7679-41b3-bfed-ea21a49be22a-kube-api-access-955b9" pod "ef992b4b-7679-41b3-bfed-ea21a49be22a" (UID: "ef992b4b-7679-41b3-bfed-ea21a49be22a"). InnerVolumeSpecName "kube-api-access-955b9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:09:30.091508 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.091433 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef992b4b-7679-41b3-bfed-ea21a49be22a-console-serving-cert" pod "ef992b4b-7679-41b3-bfed-ea21a49be22a" (UID: "ef992b4b-7679-41b3-bfed-ea21a49be22a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:09:30.091508 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.091449 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef992b4b-7679-41b3-bfed-ea21a49be22a-console-oauth-config" pod "ef992b4b-7679-41b3-bfed-ea21a49be22a" (UID: "ef992b4b-7679-41b3-bfed-ea21a49be22a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:09:30.190198 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.190164 4615 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ef992b4b-7679-41b3-bfed-ea21a49be22a-tmp\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:30.190198 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.190190 4615 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef992b4b-7679-41b3-bfed-ea21a49be22a-oauth-serving-cert\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:30.190198 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.190202 4615 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef992b4b-7679-41b3-bfed-ea21a49be22a-console-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:30.190445 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.190211 4615 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef992b4b-7679-41b3-bfed-ea21a49be22a-console-oauth-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:30.190445 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.190219 4615 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef992b4b-7679-41b3-bfed-ea21a49be22a-service-ca\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:30.190445 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.190227 4615 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef992b4b-7679-41b3-bfed-ea21a49be22a-console-serving-cert\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:30.190445 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.190235 4615 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-955b9\" (UniqueName: \"kubernetes.io/projected/ef992b4b-7679-41b3-bfed-ea21a49be22a-kube-api-access-955b9\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:30.206139 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.206103 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-54cf75fc99-cf9gr"] Oct 30 22:09:30.206409 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.206397 4615 cpu_manager.go:414] "containerMap: removing container" podUID="04f40ed9-0eef-4946-b3e6-feafcfee8e03" containerName="insights-gathering" Oct 30 22:09:30.206462 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.206410 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="04f40ed9-0eef-4946-b3e6-feafcfee8e03" containerName="insights-gathering" Oct 30 22:09:30.206462 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.206428 4615 cpu_manager.go:414] "containerMap: removing container" podUID="ef992b4b-7679-41b3-bfed-ea21a49be22a" containerName="console" Oct 30 22:09:30.206462 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.206433 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef992b4b-7679-41b3-bfed-ea21a49be22a" containerName="console" Oct 30 22:09:30.206562 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.206484 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="04f40ed9-0eef-4946-b3e6-feafcfee8e03" containerName="insights-gathering" Oct 30 22:09:30.206562 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.206492 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef992b4b-7679-41b3-bfed-ea21a49be22a" containerName="console" Oct 30 22:09:30.230274 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.230246 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.235449 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.235421 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54cf75fc99-cf9gr"] Oct 30 22:09:30.290578 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.290546 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec244118-01e7-4088-9048-e8004969bf96-trusted-ca-bundle\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.290773 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.290585 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzk7j\" (UniqueName: \"kubernetes.io/projected/ec244118-01e7-4088-9048-e8004969bf96-kube-api-access-fzk7j\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.290773 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.290616 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec244118-01e7-4088-9048-e8004969bf96-oauth-serving-cert\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.290773 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.290699 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec244118-01e7-4088-9048-e8004969bf96-service-ca\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.290916 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.290809 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec244118-01e7-4088-9048-e8004969bf96-console-oauth-config\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.290916 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.290840 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec244118-01e7-4088-9048-e8004969bf96-console-config\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.290916 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.290872 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec244118-01e7-4088-9048-e8004969bf96-console-serving-cert\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.291080 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.290970 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec244118-01e7-4088-9048-e8004969bf96-tmp\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.349158 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.349041 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:09:30.391547 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.391507 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec244118-01e7-4088-9048-e8004969bf96-console-config\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.391729 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.391555 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec244118-01e7-4088-9048-e8004969bf96-console-serving-cert\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.391729 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.391603 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec244118-01e7-4088-9048-e8004969bf96-tmp\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.391729 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.391637 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec244118-01e7-4088-9048-e8004969bf96-trusted-ca-bundle\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.391729 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.391666 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzk7j\" (UniqueName: \"kubernetes.io/projected/ec244118-01e7-4088-9048-e8004969bf96-kube-api-access-fzk7j\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.391729 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.391715 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec244118-01e7-4088-9048-e8004969bf96-oauth-serving-cert\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.392021 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.391742 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec244118-01e7-4088-9048-e8004969bf96-service-ca\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.392021 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.391894 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec244118-01e7-4088-9048-e8004969bf96-console-oauth-config\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.392165 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.392133 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec244118-01e7-4088-9048-e8004969bf96-tmp\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.392473 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.392430 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec244118-01e7-4088-9048-e8004969bf96-oauth-serving-cert\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.392473 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.392430 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec244118-01e7-4088-9048-e8004969bf96-console-config\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.392654 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.392639 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec244118-01e7-4088-9048-e8004969bf96-trusted-ca-bundle\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.392715 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.392694 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec244118-01e7-4088-9048-e8004969bf96-service-ca\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.394209 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.394178 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec244118-01e7-4088-9048-e8004969bf96-console-serving-cert\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.394359 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.394340 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec244118-01e7-4088-9048-e8004969bf96-console-oauth-config\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.413011 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.412970 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzk7j\" (UniqueName: \"kubernetes.io/projected/ec244118-01e7-4088-9048-e8004969bf96-kube-api-access-fzk7j\") pod \"console-54cf75fc99-cf9gr\" (UID: \"ec244118-01e7-4088-9048-e8004969bf96\") " pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.483874 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.483826 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:09:30.559608 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.559571 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:30.662875 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.662833 4615 generic.go:356] "Generic (PLEG): container finished" podID="ef992b4b-7679-41b3-bfed-ea21a49be22a" containerID="aea5dbec327495dcaa879e7f8b2b7ef79fc93133d6bb520648b9dd87243fe98c" exitCode=0 Oct 30 22:09:30.663115 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.662909 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9f78dd5d-nnrh8" Oct 30 22:09:30.663115 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.662926 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9f78dd5d-nnrh8" event={"ID":"ef992b4b-7679-41b3-bfed-ea21a49be22a","Type":"ContainerDied","Data":"aea5dbec327495dcaa879e7f8b2b7ef79fc93133d6bb520648b9dd87243fe98c"} Oct 30 22:09:30.663115 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.662998 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9f78dd5d-nnrh8" event={"ID":"ef992b4b-7679-41b3-bfed-ea21a49be22a","Type":"ContainerDied","Data":"e9578b4988bed5dbe79565300b830fee7314e951ec7d7b6e0cc6454e2b825b5f"} Oct 30 22:09:30.663115 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.663018 4615 scope.go:122] "RemoveContainer" containerID="aea5dbec327495dcaa879e7f8b2b7ef79fc93133d6bb520648b9dd87243fe98c" Oct 30 22:09:30.671245 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.671224 4615 scope.go:122] "RemoveContainer" containerID="aea5dbec327495dcaa879e7f8b2b7ef79fc93133d6bb520648b9dd87243fe98c" Oct 30 22:09:30.671538 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:09:30.671519 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aea5dbec327495dcaa879e7f8b2b7ef79fc93133d6bb520648b9dd87243fe98c\": container with ID starting with aea5dbec327495dcaa879e7f8b2b7ef79fc93133d6bb520648b9dd87243fe98c not found: ID does not exist" containerID="aea5dbec327495dcaa879e7f8b2b7ef79fc93133d6bb520648b9dd87243fe98c" Oct 30 22:09:30.671599 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.671547 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aea5dbec327495dcaa879e7f8b2b7ef79fc93133d6bb520648b9dd87243fe98c"} err="failed to get container status \"aea5dbec327495dcaa879e7f8b2b7ef79fc93133d6bb520648b9dd87243fe98c\": rpc error: code = NotFound desc = could not find container \"aea5dbec327495dcaa879e7f8b2b7ef79fc93133d6bb520648b9dd87243fe98c\": container with ID starting with aea5dbec327495dcaa879e7f8b2b7ef79fc93133d6bb520648b9dd87243fe98c not found: ID does not exist" Oct 30 22:09:30.672509 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.672489 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:09:30.716637 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.716602 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9f78dd5d-nnrh8"] Oct 30 22:09:30.718676 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.718650 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9f78dd5d-nnrh8"] Oct 30 22:09:30.848273 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.848244 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:09:30.875765 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:30.875737 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54cf75fc99-cf9gr"] Oct 30 22:09:30.876990 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:09:30.876954 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec244118_01e7_4088_9048_e8004969bf96.slice/crio-fc6c5a84ee21ca4d5072fc2eba76334cff1718f0c384bfb3840a8ae7003943a4 WatchSource:0}: Error finding container fc6c5a84ee21ca4d5072fc2eba76334cff1718f0c384bfb3840a8ae7003943a4: Status 404 returned error can't find the container with id fc6c5a84ee21ca4d5072fc2eba76334cff1718f0c384bfb3840a8ae7003943a4 Oct 30 22:09:31.232941 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:31.232857 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ef992b4b-7679-41b3-bfed-ea21a49be22a" path="/var/lib/kubelet/pods/ef992b4b-7679-41b3-bfed-ea21a49be22a/volumes" Oct 30 22:09:31.668523 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:31.668482 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54cf75fc99-cf9gr" event={"ID":"ec244118-01e7-4088-9048-e8004969bf96","Type":"ContainerStarted","Data":"ffc791c5ce6a70f0b29b13ff3485883accf0d6364df670fc98868b86bdde6a63"} Oct 30 22:09:31.668930 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:31.668533 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54cf75fc99-cf9gr" event={"ID":"ec244118-01e7-4088-9048-e8004969bf96","Type":"ContainerStarted","Data":"fc6c5a84ee21ca4d5072fc2eba76334cff1718f0c384bfb3840a8ae7003943a4"} Oct 30 22:09:31.722683 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:31.722626 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-54cf75fc99-cf9gr" podStartSLOduration=1.72260882 podStartE2EDuration="1.72260882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:09:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:31.710016123 +0000 UTC m=+88.625702427" watchObservedRunningTime="2026-10-30 22:09:31.72260882 +0000 UTC m=+88.638295122" Oct 30 22:09:34.576683 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:34.576647 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 22:09:34.577220 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:34.576927 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="alertmanager" containerID="cri-o://b0110e357a28e74b1477befec5370e6b7636e2f0b6e9c1d63a104db2f3d63bc6" gracePeriod=120 Oct 30 22:09:34.577220 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:34.576995 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="kube-rbac-proxy-metric" containerID="cri-o://b8d9847a7548ad3c9f2ae8cf1e38f9628613deddd40b92f1e13f8082626ea079" gracePeriod=120 Oct 30 22:09:34.577220 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:34.577028 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="kube-rbac-proxy-web" containerID="cri-o://bfd52530fc208adc75492c1da99fdd3065ea83539cecec018e88d94a48c45891" gracePeriod=120 Oct 30 22:09:34.577220 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:34.577041 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="config-reloader" containerID="cri-o://e645ec4cfd933857bb7a7b4ef26f90d9fa1d6f6c8b5183e9ed05ce893e65fa26" gracePeriod=120 Oct 30 22:09:34.577220 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:34.577101 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="prom-label-proxy" containerID="cri-o://adacc828579c0c2194152ec747945ea2782bbb58afe5372ddc5cb7a73167f069" gracePeriod=120 Oct 30 22:09:34.577220 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:34.577101 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="kube-rbac-proxy" containerID="cri-o://e4f41d220d3530a3d37ab2826d51fc120735e57c2e09ac09678c71c8fefc24db" gracePeriod=120 Oct 30 22:09:35.688693 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.688654 4615 generic.go:356] "Generic (PLEG): container finished" podID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerID="adacc828579c0c2194152ec747945ea2782bbb58afe5372ddc5cb7a73167f069" exitCode=0 Oct 30 22:09:35.688693 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.688685 4615 generic.go:356] "Generic (PLEG): container finished" podID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerID="e4f41d220d3530a3d37ab2826d51fc120735e57c2e09ac09678c71c8fefc24db" exitCode=0 Oct 30 22:09:35.688693 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.688696 4615 generic.go:356] "Generic (PLEG): container finished" podID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerID="e645ec4cfd933857bb7a7b4ef26f90d9fa1d6f6c8b5183e9ed05ce893e65fa26" exitCode=0 Oct 30 22:09:35.688693 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.688703 4615 generic.go:356] "Generic (PLEG): container finished" podID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerID="b0110e357a28e74b1477befec5370e6b7636e2f0b6e9c1d63a104db2f3d63bc6" exitCode=0 Oct 30 22:09:35.689227 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.688719 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19e30c8d-86ff-4018-814f-4f3515a38bfb","Type":"ContainerDied","Data":"adacc828579c0c2194152ec747945ea2782bbb58afe5372ddc5cb7a73167f069"} Oct 30 22:09:35.689227 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.688787 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19e30c8d-86ff-4018-814f-4f3515a38bfb","Type":"ContainerDied","Data":"e4f41d220d3530a3d37ab2826d51fc120735e57c2e09ac09678c71c8fefc24db"} Oct 30 22:09:35.689227 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.688815 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19e30c8d-86ff-4018-814f-4f3515a38bfb","Type":"ContainerDied","Data":"e645ec4cfd933857bb7a7b4ef26f90d9fa1d6f6c8b5183e9ed05ce893e65fa26"} Oct 30 22:09:35.689227 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.688841 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19e30c8d-86ff-4018-814f-4f3515a38bfb","Type":"ContainerDied","Data":"b0110e357a28e74b1477befec5370e6b7636e2f0b6e9c1d63a104db2f3d63bc6"} Oct 30 22:09:35.859093 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.859055 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:35.940175 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.940089 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-config-volume\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-config-volume\") pod \"19e30c8d-86ff-4018-814f-4f3515a38bfb\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " Oct 30 22:09:35.940175 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.940136 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19e30c8d-86ff-4018-814f-4f3515a38bfb-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19e30c8d-86ff-4018-814f-4f3515a38bfb-metrics-client-ca\") pod \"19e30c8d-86ff-4018-814f-4f3515a38bfb\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " Oct 30 22:09:35.940175 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.940168 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-web-config\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-web-config\") pod \"19e30c8d-86ff-4018-814f-4f3515a38bfb\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " Oct 30 22:09:35.940465 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.940187 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19e30c8d-86ff-4018-814f-4f3515a38bfb-config-out\" (UniqueName: \"kubernetes.io/empty-dir/19e30c8d-86ff-4018-814f-4f3515a38bfb-config-out\") pod \"19e30c8d-86ff-4018-814f-4f3515a38bfb\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " Oct 30 22:09:35.940465 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.940210 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"19e30c8d-86ff-4018-814f-4f3515a38bfb\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " Oct 30 22:09:35.940465 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.940267 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-kube-rbac-proxy\") pod \"19e30c8d-86ff-4018-814f-4f3515a38bfb\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " Oct 30 22:09:35.940465 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.940301 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-cluster-tls-config\") pod \"19e30c8d-86ff-4018-814f-4f3515a38bfb\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " Oct 30 22:09:35.940465 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.940338 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-kube-rbac-proxy-web\") pod \"19e30c8d-86ff-4018-814f-4f3515a38bfb\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " Oct 30 22:09:35.940465 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.940366 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19e30c8d-86ff-4018-814f-4f3515a38bfb-tls-assets\" (UniqueName: \"kubernetes.io/projected/19e30c8d-86ff-4018-814f-4f3515a38bfb-tls-assets\") pod \"19e30c8d-86ff-4018-814f-4f3515a38bfb\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " Oct 30 22:09:35.940465 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.940396 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-main-tls\") pod \"19e30c8d-86ff-4018-814f-4f3515a38bfb\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " Oct 30 22:09:35.940465 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.940426 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19e30c8d-86ff-4018-814f-4f3515a38bfb-kube-api-access-2g2nm\" (UniqueName: \"kubernetes.io/projected/19e30c8d-86ff-4018-814f-4f3515a38bfb-kube-api-access-2g2nm\") pod \"19e30c8d-86ff-4018-814f-4f3515a38bfb\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " Oct 30 22:09:35.940465 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.940454 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19e30c8d-86ff-4018-814f-4f3515a38bfb-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19e30c8d-86ff-4018-814f-4f3515a38bfb-alertmanager-trusted-ca-bundle\") pod \"19e30c8d-86ff-4018-814f-4f3515a38bfb\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " Oct 30 22:09:35.940929 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.940498 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19e30c8d-86ff-4018-814f-4f3515a38bfb-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19e30c8d-86ff-4018-814f-4f3515a38bfb-alertmanager-main-db\") pod \"19e30c8d-86ff-4018-814f-4f3515a38bfb\" (UID: \"19e30c8d-86ff-4018-814f-4f3515a38bfb\") " Oct 30 22:09:35.940929 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.940651 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19e30c8d-86ff-4018-814f-4f3515a38bfb-metrics-client-ca" pod "19e30c8d-86ff-4018-814f-4f3515a38bfb" (UID: "19e30c8d-86ff-4018-814f-4f3515a38bfb"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:09:35.940929 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.940770 4615 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19e30c8d-86ff-4018-814f-4f3515a38bfb-metrics-client-ca\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:35.941657 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.941525 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19e30c8d-86ff-4018-814f-4f3515a38bfb-alertmanager-trusted-ca-bundle" pod "19e30c8d-86ff-4018-814f-4f3515a38bfb" (UID: "19e30c8d-86ff-4018-814f-4f3515a38bfb"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:09:35.942711 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.942674 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19e30c8d-86ff-4018-814f-4f3515a38bfb-alertmanager-main-db" pod "19e30c8d-86ff-4018-814f-4f3515a38bfb" (UID: "19e30c8d-86ff-4018-814f-4f3515a38bfb"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:09:35.944385 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.944359 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-config-volume" pod "19e30c8d-86ff-4018-814f-4f3515a38bfb" (UID: "19e30c8d-86ff-4018-814f-4f3515a38bfb"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:09:35.944385 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.944369 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19e30c8d-86ff-4018-814f-4f3515a38bfb-config-out" pod "19e30c8d-86ff-4018-814f-4f3515a38bfb" (UID: "19e30c8d-86ff-4018-814f-4f3515a38bfb"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:09:35.944569 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.944405 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-kube-rbac-proxy-metric" pod "19e30c8d-86ff-4018-814f-4f3515a38bfb" (UID: "19e30c8d-86ff-4018-814f-4f3515a38bfb"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:09:35.944628 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.944601 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-main-tls" pod "19e30c8d-86ff-4018-814f-4f3515a38bfb" (UID: "19e30c8d-86ff-4018-814f-4f3515a38bfb"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:09:35.944754 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.944722 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-kube-rbac-proxy-web" pod "19e30c8d-86ff-4018-814f-4f3515a38bfb" (UID: "19e30c8d-86ff-4018-814f-4f3515a38bfb"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:09:35.945074 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.945029 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-kube-rbac-proxy" pod "19e30c8d-86ff-4018-814f-4f3515a38bfb" (UID: "19e30c8d-86ff-4018-814f-4f3515a38bfb"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:09:35.945175 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.945073 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19e30c8d-86ff-4018-814f-4f3515a38bfb-kube-api-access-2g2nm" pod "19e30c8d-86ff-4018-814f-4f3515a38bfb" (UID: "19e30c8d-86ff-4018-814f-4f3515a38bfb"). InnerVolumeSpecName "kube-api-access-2g2nm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:09:35.945175 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.945091 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19e30c8d-86ff-4018-814f-4f3515a38bfb-tls-assets" pod "19e30c8d-86ff-4018-814f-4f3515a38bfb" (UID: "19e30c8d-86ff-4018-814f-4f3515a38bfb"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:09:35.947650 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.947623 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-cluster-tls-config" pod "19e30c8d-86ff-4018-814f-4f3515a38bfb" (UID: "19e30c8d-86ff-4018-814f-4f3515a38bfb"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:09:35.953986 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:35.953960 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-web-config" pod "19e30c8d-86ff-4018-814f-4f3515a38bfb" (UID: "19e30c8d-86ff-4018-814f-4f3515a38bfb"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:09:36.041947 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.041907 4615 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-web-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:36.041947 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.041938 4615 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19e30c8d-86ff-4018-814f-4f3515a38bfb-config-out\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:36.041947 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.041949 4615 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:36.042195 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.041958 4615 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:36.042195 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.041969 4615 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-cluster-tls-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:36.042195 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.041978 4615 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:36.042195 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.041986 4615 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19e30c8d-86ff-4018-814f-4f3515a38bfb-tls-assets\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:36.042195 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.041995 4615 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-secret-alertmanager-main-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:36.042195 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.042004 4615 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2g2nm\" (UniqueName: \"kubernetes.io/projected/19e30c8d-86ff-4018-814f-4f3515a38bfb-kube-api-access-2g2nm\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:36.042195 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.042012 4615 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19e30c8d-86ff-4018-814f-4f3515a38bfb-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:36.042195 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.042020 4615 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19e30c8d-86ff-4018-814f-4f3515a38bfb-alertmanager-main-db\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:36.042195 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.042027 4615 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19e30c8d-86ff-4018-814f-4f3515a38bfb-config-volume\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:09:36.700127 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.700091 4615 generic.go:356] "Generic (PLEG): container finished" podID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerID="b8d9847a7548ad3c9f2ae8cf1e38f9628613deddd40b92f1e13f8082626ea079" exitCode=0 Oct 30 22:09:36.700127 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.700119 4615 generic.go:356] "Generic (PLEG): container finished" podID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerID="bfd52530fc208adc75492c1da99fdd3065ea83539cecec018e88d94a48c45891" exitCode=0 Oct 30 22:09:36.700577 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.700173 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19e30c8d-86ff-4018-814f-4f3515a38bfb","Type":"ContainerDied","Data":"b8d9847a7548ad3c9f2ae8cf1e38f9628613deddd40b92f1e13f8082626ea079"} Oct 30 22:09:36.700577 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.700184 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:36.700577 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.700276 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19e30c8d-86ff-4018-814f-4f3515a38bfb","Type":"ContainerDied","Data":"bfd52530fc208adc75492c1da99fdd3065ea83539cecec018e88d94a48c45891"} Oct 30 22:09:36.700577 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.700307 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19e30c8d-86ff-4018-814f-4f3515a38bfb","Type":"ContainerDied","Data":"15a5e4d44a1ee727f17aabd3906a8dee400901aa306cceb3ee3d3c6e684f2459"} Oct 30 22:09:36.700577 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.700342 4615 scope.go:122] "RemoveContainer" containerID="adacc828579c0c2194152ec747945ea2782bbb58afe5372ddc5cb7a73167f069" Oct 30 22:09:36.708686 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.708665 4615 scope.go:122] "RemoveContainer" containerID="b8d9847a7548ad3c9f2ae8cf1e38f9628613deddd40b92f1e13f8082626ea079" Oct 30 22:09:36.715800 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.715782 4615 scope.go:122] "RemoveContainer" containerID="e4f41d220d3530a3d37ab2826d51fc120735e57c2e09ac09678c71c8fefc24db" Oct 30 22:09:36.722807 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.722783 4615 scope.go:122] "RemoveContainer" containerID="bfd52530fc208adc75492c1da99fdd3065ea83539cecec018e88d94a48c45891" Oct 30 22:09:36.729467 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.729446 4615 scope.go:122] "RemoveContainer" containerID="e645ec4cfd933857bb7a7b4ef26f90d9fa1d6f6c8b5183e9ed05ce893e65fa26" Oct 30 22:09:36.736349 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.736327 4615 scope.go:122] "RemoveContainer" containerID="b0110e357a28e74b1477befec5370e6b7636e2f0b6e9c1d63a104db2f3d63bc6" Oct 30 22:09:36.743271 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.743242 4615 scope.go:122] "RemoveContainer" containerID="39eb90131d7672ec1b9b06c56fd0baba6f178647349316af9c6b710563b6586f" Oct 30 22:09:36.750156 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.750140 4615 scope.go:122] "RemoveContainer" containerID="adacc828579c0c2194152ec747945ea2782bbb58afe5372ddc5cb7a73167f069" Oct 30 22:09:36.750443 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:09:36.750423 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adacc828579c0c2194152ec747945ea2782bbb58afe5372ddc5cb7a73167f069\": container with ID starting with adacc828579c0c2194152ec747945ea2782bbb58afe5372ddc5cb7a73167f069 not found: ID does not exist" containerID="adacc828579c0c2194152ec747945ea2782bbb58afe5372ddc5cb7a73167f069" Oct 30 22:09:36.750495 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.750452 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adacc828579c0c2194152ec747945ea2782bbb58afe5372ddc5cb7a73167f069"} err="failed to get container status \"adacc828579c0c2194152ec747945ea2782bbb58afe5372ddc5cb7a73167f069\": rpc error: code = NotFound desc = could not find container \"adacc828579c0c2194152ec747945ea2782bbb58afe5372ddc5cb7a73167f069\": container with ID starting with adacc828579c0c2194152ec747945ea2782bbb58afe5372ddc5cb7a73167f069 not found: ID does not exist" Oct 30 22:09:36.750495 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.750471 4615 scope.go:122] "RemoveContainer" containerID="b8d9847a7548ad3c9f2ae8cf1e38f9628613deddd40b92f1e13f8082626ea079" Oct 30 22:09:36.750723 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:09:36.750705 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8d9847a7548ad3c9f2ae8cf1e38f9628613deddd40b92f1e13f8082626ea079\": container with ID starting with b8d9847a7548ad3c9f2ae8cf1e38f9628613deddd40b92f1e13f8082626ea079 not found: ID does not exist" containerID="b8d9847a7548ad3c9f2ae8cf1e38f9628613deddd40b92f1e13f8082626ea079" Oct 30 22:09:36.750787 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.750732 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8d9847a7548ad3c9f2ae8cf1e38f9628613deddd40b92f1e13f8082626ea079"} err="failed to get container status \"b8d9847a7548ad3c9f2ae8cf1e38f9628613deddd40b92f1e13f8082626ea079\": rpc error: code = NotFound desc = could not find container \"b8d9847a7548ad3c9f2ae8cf1e38f9628613deddd40b92f1e13f8082626ea079\": container with ID starting with b8d9847a7548ad3c9f2ae8cf1e38f9628613deddd40b92f1e13f8082626ea079 not found: ID does not exist" Oct 30 22:09:36.750787 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.750754 4615 scope.go:122] "RemoveContainer" containerID="e4f41d220d3530a3d37ab2826d51fc120735e57c2e09ac09678c71c8fefc24db" Oct 30 22:09:36.751001 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:09:36.750985 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4f41d220d3530a3d37ab2826d51fc120735e57c2e09ac09678c71c8fefc24db\": container with ID starting with e4f41d220d3530a3d37ab2826d51fc120735e57c2e09ac09678c71c8fefc24db not found: ID does not exist" containerID="e4f41d220d3530a3d37ab2826d51fc120735e57c2e09ac09678c71c8fefc24db" Oct 30 22:09:36.751078 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.751008 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4f41d220d3530a3d37ab2826d51fc120735e57c2e09ac09678c71c8fefc24db"} err="failed to get container status \"e4f41d220d3530a3d37ab2826d51fc120735e57c2e09ac09678c71c8fefc24db\": rpc error: code = NotFound desc = could not find container \"e4f41d220d3530a3d37ab2826d51fc120735e57c2e09ac09678c71c8fefc24db\": container with ID starting with e4f41d220d3530a3d37ab2826d51fc120735e57c2e09ac09678c71c8fefc24db not found: ID does not exist" Oct 30 22:09:36.751078 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.751029 4615 scope.go:122] "RemoveContainer" containerID="bfd52530fc208adc75492c1da99fdd3065ea83539cecec018e88d94a48c45891" Oct 30 22:09:36.751281 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:09:36.751264 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfd52530fc208adc75492c1da99fdd3065ea83539cecec018e88d94a48c45891\": container with ID starting with bfd52530fc208adc75492c1da99fdd3065ea83539cecec018e88d94a48c45891 not found: ID does not exist" containerID="bfd52530fc208adc75492c1da99fdd3065ea83539cecec018e88d94a48c45891" Oct 30 22:09:36.751318 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.751286 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfd52530fc208adc75492c1da99fdd3065ea83539cecec018e88d94a48c45891"} err="failed to get container status \"bfd52530fc208adc75492c1da99fdd3065ea83539cecec018e88d94a48c45891\": rpc error: code = NotFound desc = could not find container \"bfd52530fc208adc75492c1da99fdd3065ea83539cecec018e88d94a48c45891\": container with ID starting with bfd52530fc208adc75492c1da99fdd3065ea83539cecec018e88d94a48c45891 not found: ID does not exist" Oct 30 22:09:36.751318 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.751301 4615 scope.go:122] "RemoveContainer" containerID="e645ec4cfd933857bb7a7b4ef26f90d9fa1d6f6c8b5183e9ed05ce893e65fa26" Oct 30 22:09:36.751539 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:09:36.751522 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e645ec4cfd933857bb7a7b4ef26f90d9fa1d6f6c8b5183e9ed05ce893e65fa26\": container with ID starting with e645ec4cfd933857bb7a7b4ef26f90d9fa1d6f6c8b5183e9ed05ce893e65fa26 not found: ID does not exist" containerID="e645ec4cfd933857bb7a7b4ef26f90d9fa1d6f6c8b5183e9ed05ce893e65fa26" Oct 30 22:09:36.751610 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.751546 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e645ec4cfd933857bb7a7b4ef26f90d9fa1d6f6c8b5183e9ed05ce893e65fa26"} err="failed to get container status \"e645ec4cfd933857bb7a7b4ef26f90d9fa1d6f6c8b5183e9ed05ce893e65fa26\": rpc error: code = NotFound desc = could not find container \"e645ec4cfd933857bb7a7b4ef26f90d9fa1d6f6c8b5183e9ed05ce893e65fa26\": container with ID starting with e645ec4cfd933857bb7a7b4ef26f90d9fa1d6f6c8b5183e9ed05ce893e65fa26 not found: ID does not exist" Oct 30 22:09:36.751610 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.751566 4615 scope.go:122] "RemoveContainer" containerID="b0110e357a28e74b1477befec5370e6b7636e2f0b6e9c1d63a104db2f3d63bc6" Oct 30 22:09:36.751782 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:09:36.751765 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0110e357a28e74b1477befec5370e6b7636e2f0b6e9c1d63a104db2f3d63bc6\": container with ID starting with b0110e357a28e74b1477befec5370e6b7636e2f0b6e9c1d63a104db2f3d63bc6 not found: ID does not exist" containerID="b0110e357a28e74b1477befec5370e6b7636e2f0b6e9c1d63a104db2f3d63bc6" Oct 30 22:09:36.751818 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.751786 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0110e357a28e74b1477befec5370e6b7636e2f0b6e9c1d63a104db2f3d63bc6"} err="failed to get container status \"b0110e357a28e74b1477befec5370e6b7636e2f0b6e9c1d63a104db2f3d63bc6\": rpc error: code = NotFound desc = could not find container \"b0110e357a28e74b1477befec5370e6b7636e2f0b6e9c1d63a104db2f3d63bc6\": container with ID starting with b0110e357a28e74b1477befec5370e6b7636e2f0b6e9c1d63a104db2f3d63bc6 not found: ID does not exist" Oct 30 22:09:36.751818 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.751798 4615 scope.go:122] "RemoveContainer" containerID="39eb90131d7672ec1b9b06c56fd0baba6f178647349316af9c6b710563b6586f" Oct 30 22:09:36.752027 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:09:36.752010 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39eb90131d7672ec1b9b06c56fd0baba6f178647349316af9c6b710563b6586f\": container with ID starting with 39eb90131d7672ec1b9b06c56fd0baba6f178647349316af9c6b710563b6586f not found: ID does not exist" containerID="39eb90131d7672ec1b9b06c56fd0baba6f178647349316af9c6b710563b6586f" Oct 30 22:09:36.752104 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.752034 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39eb90131d7672ec1b9b06c56fd0baba6f178647349316af9c6b710563b6586f"} err="failed to get container status \"39eb90131d7672ec1b9b06c56fd0baba6f178647349316af9c6b710563b6586f\": rpc error: code = NotFound desc = could not find container \"39eb90131d7672ec1b9b06c56fd0baba6f178647349316af9c6b710563b6586f\": container with ID starting with 39eb90131d7672ec1b9b06c56fd0baba6f178647349316af9c6b710563b6586f not found: ID does not exist" Oct 30 22:09:36.752104 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.752073 4615 scope.go:122] "RemoveContainer" containerID="adacc828579c0c2194152ec747945ea2782bbb58afe5372ddc5cb7a73167f069" Oct 30 22:09:36.752324 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.752307 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adacc828579c0c2194152ec747945ea2782bbb58afe5372ddc5cb7a73167f069"} err="failed to get container status \"adacc828579c0c2194152ec747945ea2782bbb58afe5372ddc5cb7a73167f069\": rpc error: code = NotFound desc = could not find container \"adacc828579c0c2194152ec747945ea2782bbb58afe5372ddc5cb7a73167f069\": container with ID starting with adacc828579c0c2194152ec747945ea2782bbb58afe5372ddc5cb7a73167f069 not found: ID does not exist" Oct 30 22:09:36.752391 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.752325 4615 scope.go:122] "RemoveContainer" containerID="b8d9847a7548ad3c9f2ae8cf1e38f9628613deddd40b92f1e13f8082626ea079" Oct 30 22:09:36.752540 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.752523 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8d9847a7548ad3c9f2ae8cf1e38f9628613deddd40b92f1e13f8082626ea079"} err="failed to get container status \"b8d9847a7548ad3c9f2ae8cf1e38f9628613deddd40b92f1e13f8082626ea079\": rpc error: code = NotFound desc = could not find container \"b8d9847a7548ad3c9f2ae8cf1e38f9628613deddd40b92f1e13f8082626ea079\": container with ID starting with b8d9847a7548ad3c9f2ae8cf1e38f9628613deddd40b92f1e13f8082626ea079 not found: ID does not exist" Oct 30 22:09:36.752606 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.752541 4615 scope.go:122] "RemoveContainer" containerID="e4f41d220d3530a3d37ab2826d51fc120735e57c2e09ac09678c71c8fefc24db" Oct 30 22:09:36.752757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.752740 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4f41d220d3530a3d37ab2826d51fc120735e57c2e09ac09678c71c8fefc24db"} err="failed to get container status \"e4f41d220d3530a3d37ab2826d51fc120735e57c2e09ac09678c71c8fefc24db\": rpc error: code = NotFound desc = could not find container \"e4f41d220d3530a3d37ab2826d51fc120735e57c2e09ac09678c71c8fefc24db\": container with ID starting with e4f41d220d3530a3d37ab2826d51fc120735e57c2e09ac09678c71c8fefc24db not found: ID does not exist" Oct 30 22:09:36.752818 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.752758 4615 scope.go:122] "RemoveContainer" containerID="bfd52530fc208adc75492c1da99fdd3065ea83539cecec018e88d94a48c45891" Oct 30 22:09:36.752942 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.752925 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfd52530fc208adc75492c1da99fdd3065ea83539cecec018e88d94a48c45891"} err="failed to get container status \"bfd52530fc208adc75492c1da99fdd3065ea83539cecec018e88d94a48c45891\": rpc error: code = NotFound desc = could not find container \"bfd52530fc208adc75492c1da99fdd3065ea83539cecec018e88d94a48c45891\": container with ID starting with bfd52530fc208adc75492c1da99fdd3065ea83539cecec018e88d94a48c45891 not found: ID does not exist" Oct 30 22:09:36.753015 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.752943 4615 scope.go:122] "RemoveContainer" containerID="e645ec4cfd933857bb7a7b4ef26f90d9fa1d6f6c8b5183e9ed05ce893e65fa26" Oct 30 22:09:36.753182 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.753162 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e645ec4cfd933857bb7a7b4ef26f90d9fa1d6f6c8b5183e9ed05ce893e65fa26"} err="failed to get container status \"e645ec4cfd933857bb7a7b4ef26f90d9fa1d6f6c8b5183e9ed05ce893e65fa26\": rpc error: code = NotFound desc = could not find container \"e645ec4cfd933857bb7a7b4ef26f90d9fa1d6f6c8b5183e9ed05ce893e65fa26\": container with ID starting with e645ec4cfd933857bb7a7b4ef26f90d9fa1d6f6c8b5183e9ed05ce893e65fa26 not found: ID does not exist" Oct 30 22:09:36.753254 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.753184 4615 scope.go:122] "RemoveContainer" containerID="b0110e357a28e74b1477befec5370e6b7636e2f0b6e9c1d63a104db2f3d63bc6" Oct 30 22:09:36.753393 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.753377 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0110e357a28e74b1477befec5370e6b7636e2f0b6e9c1d63a104db2f3d63bc6"} err="failed to get container status \"b0110e357a28e74b1477befec5370e6b7636e2f0b6e9c1d63a104db2f3d63bc6\": rpc error: code = NotFound desc = could not find container \"b0110e357a28e74b1477befec5370e6b7636e2f0b6e9c1d63a104db2f3d63bc6\": container with ID starting with b0110e357a28e74b1477befec5370e6b7636e2f0b6e9c1d63a104db2f3d63bc6 not found: ID does not exist" Oct 30 22:09:36.753460 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.753395 4615 scope.go:122] "RemoveContainer" containerID="39eb90131d7672ec1b9b06c56fd0baba6f178647349316af9c6b710563b6586f" Oct 30 22:09:36.753626 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.753608 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39eb90131d7672ec1b9b06c56fd0baba6f178647349316af9c6b710563b6586f"} err="failed to get container status \"39eb90131d7672ec1b9b06c56fd0baba6f178647349316af9c6b710563b6586f\": rpc error: code = NotFound desc = could not find container \"39eb90131d7672ec1b9b06c56fd0baba6f178647349316af9c6b710563b6586f\": container with ID starting with 39eb90131d7672ec1b9b06c56fd0baba6f178647349316af9c6b710563b6586f not found: ID does not exist" Oct 30 22:09:36.768477 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.768443 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 22:09:36.781326 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.781299 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 22:09:36.804769 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.804739 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 22:09:36.805011 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.804998 4615 cpu_manager.go:414] "containerMap: removing container" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="prom-label-proxy" Oct 30 22:09:36.805011 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.805011 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="prom-label-proxy" Oct 30 22:09:36.805186 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.805028 4615 cpu_manager.go:414] "containerMap: removing container" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="kube-rbac-proxy" Oct 30 22:09:36.805186 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.805033 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="kube-rbac-proxy" Oct 30 22:09:36.805186 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.805068 4615 cpu_manager.go:414] "containerMap: removing container" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="init-config-reloader" Oct 30 22:09:36.805186 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.805076 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="init-config-reloader" Oct 30 22:09:36.805186 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.805087 4615 cpu_manager.go:414] "containerMap: removing container" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="config-reloader" Oct 30 22:09:36.805186 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.805093 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="config-reloader" Oct 30 22:09:36.805186 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.805100 4615 cpu_manager.go:414] "containerMap: removing container" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="kube-rbac-proxy-metric" Oct 30 22:09:36.805186 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.805105 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="kube-rbac-proxy-metric" Oct 30 22:09:36.805186 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.805112 4615 cpu_manager.go:414] "containerMap: removing container" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="kube-rbac-proxy-web" Oct 30 22:09:36.805186 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.805116 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="kube-rbac-proxy-web" Oct 30 22:09:36.805186 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.805129 4615 cpu_manager.go:414] "containerMap: removing container" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="alertmanager" Oct 30 22:09:36.805186 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.805134 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="alertmanager" Oct 30 22:09:36.805186 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.805184 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="kube-rbac-proxy" Oct 30 22:09:36.805186 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.805192 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="config-reloader" Oct 30 22:09:36.805186 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.805198 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="kube-rbac-proxy-metric" Oct 30 22:09:36.805710 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.805204 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="kube-rbac-proxy-web" Oct 30 22:09:36.805710 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.805211 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="alertmanager" Oct 30 22:09:36.805710 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.805217 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" containerName="prom-label-proxy" Oct 30 22:09:36.840177 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.840139 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 22:09:36.840343 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.840263 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:36.858399 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.858373 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Oct 30 22:09:36.858673 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.858650 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Oct 30 22:09:36.858786 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.858688 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-sxkkv\"" Oct 30 22:09:36.858786 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.858688 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Oct 30 22:09:36.858786 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.858695 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Oct 30 22:09:36.858786 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.858722 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Oct 30 22:09:36.858786 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.858733 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Oct 30 22:09:36.858786 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.858742 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Oct 30 22:09:36.859062 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.858984 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Oct 30 22:09:36.871436 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.871410 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Oct 30 22:09:36.947231 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.947146 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3191aa1c-021d-47b1-82ac-0edd9e17325b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:36.947231 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.947184 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3191aa1c-021d-47b1-82ac-0edd9e17325b-config-out\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:36.947231 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.947205 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3191aa1c-021d-47b1-82ac-0edd9e17325b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:36.947496 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.947273 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3191aa1c-021d-47b1-82ac-0edd9e17325b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:36.947496 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.947330 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3191aa1c-021d-47b1-82ac-0edd9e17325b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:36.947496 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.947360 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3191aa1c-021d-47b1-82ac-0edd9e17325b-web-config\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:36.947496 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.947380 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3191aa1c-021d-47b1-82ac-0edd9e17325b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:36.947496 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.947493 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3191aa1c-021d-47b1-82ac-0edd9e17325b-config-volume\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:36.947676 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.947526 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3191aa1c-021d-47b1-82ac-0edd9e17325b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:36.947676 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.947610 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3191aa1c-021d-47b1-82ac-0edd9e17325b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:36.947676 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.947629 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3191aa1c-021d-47b1-82ac-0edd9e17325b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:36.947676 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.947646 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76dr8\" (UniqueName: \"kubernetes.io/projected/3191aa1c-021d-47b1-82ac-0edd9e17325b-kube-api-access-76dr8\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:36.947676 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:36.947670 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3191aa1c-021d-47b1-82ac-0edd9e17325b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.048214 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.048175 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3191aa1c-021d-47b1-82ac-0edd9e17325b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.048214 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.048215 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3191aa1c-021d-47b1-82ac-0edd9e17325b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.048464 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.048231 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76dr8\" (UniqueName: \"kubernetes.io/projected/3191aa1c-021d-47b1-82ac-0edd9e17325b-kube-api-access-76dr8\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.048464 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.048253 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3191aa1c-021d-47b1-82ac-0edd9e17325b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.048464 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.048274 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3191aa1c-021d-47b1-82ac-0edd9e17325b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.048464 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.048295 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3191aa1c-021d-47b1-82ac-0edd9e17325b-config-out\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.048464 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.048318 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3191aa1c-021d-47b1-82ac-0edd9e17325b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.048464 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.048341 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3191aa1c-021d-47b1-82ac-0edd9e17325b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.048464 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.048379 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3191aa1c-021d-47b1-82ac-0edd9e17325b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.048464 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.048407 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3191aa1c-021d-47b1-82ac-0edd9e17325b-web-config\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.048464 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.048432 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3191aa1c-021d-47b1-82ac-0edd9e17325b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.048980 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.048475 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3191aa1c-021d-47b1-82ac-0edd9e17325b-config-volume\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.048980 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.048497 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3191aa1c-021d-47b1-82ac-0edd9e17325b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.048980 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.048657 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3191aa1c-021d-47b1-82ac-0edd9e17325b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.049458 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.049395 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3191aa1c-021d-47b1-82ac-0edd9e17325b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.049570 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.049457 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3191aa1c-021d-47b1-82ac-0edd9e17325b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.051279 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.051235 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3191aa1c-021d-47b1-82ac-0edd9e17325b-config-out\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.051526 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.051500 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3191aa1c-021d-47b1-82ac-0edd9e17325b-web-config\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.051611 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.051506 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3191aa1c-021d-47b1-82ac-0edd9e17325b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.052031 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.052005 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3191aa1c-021d-47b1-82ac-0edd9e17325b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.052207 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.052188 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3191aa1c-021d-47b1-82ac-0edd9e17325b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.052301 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.052280 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3191aa1c-021d-47b1-82ac-0edd9e17325b-config-volume\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.052337 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.052290 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3191aa1c-021d-47b1-82ac-0edd9e17325b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.052422 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.052398 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3191aa1c-021d-47b1-82ac-0edd9e17325b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.053531 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.053514 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3191aa1c-021d-47b1-82ac-0edd9e17325b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.069021 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.068994 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76dr8\" (UniqueName: \"kubernetes.io/projected/3191aa1c-021d-47b1-82ac-0edd9e17325b-kube-api-access-76dr8\") pod \"alertmanager-main-0\" (UID: \"3191aa1c-021d-47b1-82ac-0edd9e17325b\") " pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.232737 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.232663 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="19e30c8d-86ff-4018-814f-4f3515a38bfb" path="/var/lib/kubelet/pods/19e30c8d-86ff-4018-814f-4f3515a38bfb/volumes" Oct 30 22:09:37.241826 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.241796 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Oct 30 22:09:37.340066 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.339999 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:09:37.524282 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.524205 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:09:37.583757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.583723 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Oct 30 22:09:37.595604 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:09:37.595570 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3191aa1c_021d_47b1_82ac_0edd9e17325b.slice/crio-c0caa3b4446633a420ff86e0a57ddf43f0be4cec21ef6e0400732215b25e08a3 WatchSource:0}: Error finding container c0caa3b4446633a420ff86e0a57ddf43f0be4cec21ef6e0400732215b25e08a3: Status 404 returned error can't find the container with id c0caa3b4446633a420ff86e0a57ddf43f0be4cec21ef6e0400732215b25e08a3 Oct 30 22:09:37.705702 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.705665 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3191aa1c-021d-47b1-82ac-0edd9e17325b","Type":"ContainerStarted","Data":"4ac4b72a1a03be9120f4b3f44cba6c473cedf70ba3e782495973fc91d689ee0e"} Oct 30 22:09:37.706126 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:37.705743 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3191aa1c-021d-47b1-82ac-0edd9e17325b","Type":"ContainerStarted","Data":"c0caa3b4446633a420ff86e0a57ddf43f0be4cec21ef6e0400732215b25e08a3"} Oct 30 22:09:38.711742 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:38.711709 4615 generic.go:356] "Generic (PLEG): container finished" podID="3191aa1c-021d-47b1-82ac-0edd9e17325b" containerID="4ac4b72a1a03be9120f4b3f44cba6c473cedf70ba3e782495973fc91d689ee0e" exitCode=0 Oct 30 22:09:38.712155 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:38.711781 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3191aa1c-021d-47b1-82ac-0edd9e17325b","Type":"ContainerDied","Data":"4ac4b72a1a03be9120f4b3f44cba6c473cedf70ba3e782495973fc91d689ee0e"} Oct 30 22:09:39.725416 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:39.725385 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3191aa1c-021d-47b1-82ac-0edd9e17325b","Type":"ContainerStarted","Data":"a1a3bb7167b3619ff8a4749214a8197056893141b86d0f1432444c33607a0c24"} Oct 30 22:09:39.725869 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:39.725448 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3191aa1c-021d-47b1-82ac-0edd9e17325b","Type":"ContainerStarted","Data":"eca000c7e1ddec3bcdc4acaa41bc40637e1846c97c1e89144011f9f43d941fda"} Oct 30 22:09:39.725869 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:39.725479 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3191aa1c-021d-47b1-82ac-0edd9e17325b","Type":"ContainerStarted","Data":"a09b599fda1590138b16822bac84a305ab26af0a88ced5ae509cc33c0dac34ac"} Oct 30 22:09:39.725869 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:39.725503 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3191aa1c-021d-47b1-82ac-0edd9e17325b","Type":"ContainerStarted","Data":"99585431141d4b139c4ac68475254d27fed44882225c76ba639e1f5d613df856"} Oct 30 22:09:39.725869 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:39.725532 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3191aa1c-021d-47b1-82ac-0edd9e17325b","Type":"ContainerStarted","Data":"7b3ebb7b0007cbab85e7d2fb9932c2edfc654228d31a52af7d9276e2ce49cb7a"} Oct 30 22:09:39.725869 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:39.725556 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3191aa1c-021d-47b1-82ac-0edd9e17325b","Type":"ContainerStarted","Data":"fcd4cb1732d81571cda2c4c5a60b981057841759ab17297bd79efeeb7d06c8f5"} Oct 30 22:09:39.806720 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:39.806656 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.7938249219999998 podStartE2EDuration="3.806636133s" totalImagesPullingTime="0s" totalInitContainerRuntime="12.811211ms" isStatefulPod=false podCreationTimestamp="2026-10-30 22:09:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:09:39.792844709 +0000 UTC m=+96.708531015" watchObservedRunningTime="2026-10-30 22:09:39.806636133 +0000 UTC m=+96.722322436" Oct 30 22:09:40.569072 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:40.567213 4615 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:40.569072 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:40.567293 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:40.573283 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:40.573255 4615 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:40.732962 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:40.732933 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-54cf75fc99-cf9gr" Oct 30 22:09:40.808249 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:40.808216 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68c99c9bdd-h582d"] Oct 30 22:09:41.507205 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:41.507163 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:09:41.641572 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:41.641521 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:09:48.429888 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:09:48.429852 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:09:59.763274 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:09:59.763232 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:10:05.827905 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:05.827840 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-68c99c9bdd-h582d" podUID="cf0c1e02-8817-4c08-a9c1-38fb654e6761" containerName="console" containerID="cri-o://71839e2963698832bbbb37a5eb2e2523a2c75b641ca6511a44ef748705cebbf5" gracePeriod=15 Oct 30 22:10:06.072189 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.072159 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:10:06.098252 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.098177 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-trusted-ca-bundle\") pod \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " Oct 30 22:10:06.098252 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.098223 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-console-config\" (UniqueName: \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-console-config\") pod \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " Oct 30 22:10:06.098437 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.098256 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf0c1e02-8817-4c08-a9c1-38fb654e6761-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cf0c1e02-8817-4c08-a9c1-38fb654e6761-console-serving-cert\") pod \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " Oct 30 22:10:06.098437 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.098417 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-oauth-serving-cert\") pod \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " Oct 30 22:10:06.098533 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.098481 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-service-ca\") pod \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " Oct 30 22:10:06.098587 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.098531 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf0c1e02-8817-4c08-a9c1-38fb654e6761-kube-api-access-5hb68\" (UniqueName: \"kubernetes.io/projected/cf0c1e02-8817-4c08-a9c1-38fb654e6761-kube-api-access-5hb68\") pod \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " Oct 30 22:10:06.098587 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.098572 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf0c1e02-8817-4c08-a9c1-38fb654e6761-tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf0c1e02-8817-4c08-a9c1-38fb654e6761-tmp\") pod \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " Oct 30 22:10:06.098587 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.098574 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-trusted-ca-bundle" pod "cf0c1e02-8817-4c08-a9c1-38fb654e6761" (UID: "cf0c1e02-8817-4c08-a9c1-38fb654e6761"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:10:06.098738 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.098706 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf0c1e02-8817-4c08-a9c1-38fb654e6761-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cf0c1e02-8817-4c08-a9c1-38fb654e6761-console-oauth-config\") pod \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\" (UID: \"cf0c1e02-8817-4c08-a9c1-38fb654e6761\") " Oct 30 22:10:06.098797 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.098642 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-console-config" pod "cf0c1e02-8817-4c08-a9c1-38fb654e6761" (UID: "cf0c1e02-8817-4c08-a9c1-38fb654e6761"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:10:06.098852 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.098811 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf0c1e02-8817-4c08-a9c1-38fb654e6761-tmp" pod "cf0c1e02-8817-4c08-a9c1-38fb654e6761" (UID: "cf0c1e02-8817-4c08-a9c1-38fb654e6761"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:10:06.098852 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.098824 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-service-ca" pod "cf0c1e02-8817-4c08-a9c1-38fb654e6761" (UID: "cf0c1e02-8817-4c08-a9c1-38fb654e6761"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:10:06.098952 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.098845 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-oauth-serving-cert" pod "cf0c1e02-8817-4c08-a9c1-38fb654e6761" (UID: "cf0c1e02-8817-4c08-a9c1-38fb654e6761"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:10:06.099003 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.098988 4615 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-trusted-ca-bundle\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:10:06.099075 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.099009 4615 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-console-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:10:06.099075 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.099024 4615 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-oauth-serving-cert\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:10:06.099075 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.099038 4615 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cf0c1e02-8817-4c08-a9c1-38fb654e6761-service-ca\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:10:06.099189 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.099075 4615 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf0c1e02-8817-4c08-a9c1-38fb654e6761-tmp\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:10:06.100637 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.100609 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf0c1e02-8817-4c08-a9c1-38fb654e6761-console-oauth-config" pod "cf0c1e02-8817-4c08-a9c1-38fb654e6761" (UID: "cf0c1e02-8817-4c08-a9c1-38fb654e6761"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:10:06.100747 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.100638 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf0c1e02-8817-4c08-a9c1-38fb654e6761-console-serving-cert" pod "cf0c1e02-8817-4c08-a9c1-38fb654e6761" (UID: "cf0c1e02-8817-4c08-a9c1-38fb654e6761"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:10:06.100747 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.100639 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf0c1e02-8817-4c08-a9c1-38fb654e6761-kube-api-access-5hb68" pod "cf0c1e02-8817-4c08-a9c1-38fb654e6761" (UID: "cf0c1e02-8817-4c08-a9c1-38fb654e6761"). InnerVolumeSpecName "kube-api-access-5hb68". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:10:06.200286 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.200250 4615 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cf0c1e02-8817-4c08-a9c1-38fb654e6761-console-serving-cert\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:10:06.200286 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.200281 4615 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5hb68\" (UniqueName: \"kubernetes.io/projected/cf0c1e02-8817-4c08-a9c1-38fb654e6761-kube-api-access-5hb68\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:10:06.200286 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.200291 4615 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cf0c1e02-8817-4c08-a9c1-38fb654e6761-console-oauth-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:10:06.830296 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.830258 4615 generic.go:356] "Generic (PLEG): container finished" podID="cf0c1e02-8817-4c08-a9c1-38fb654e6761" containerID="71839e2963698832bbbb37a5eb2e2523a2c75b641ca6511a44ef748705cebbf5" exitCode=0 Oct 30 22:10:06.830742 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.830313 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68c99c9bdd-h582d" Oct 30 22:10:06.830742 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.830347 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68c99c9bdd-h582d" event={"ID":"cf0c1e02-8817-4c08-a9c1-38fb654e6761","Type":"ContainerDied","Data":"71839e2963698832bbbb37a5eb2e2523a2c75b641ca6511a44ef748705cebbf5"} Oct 30 22:10:06.830742 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.830401 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68c99c9bdd-h582d" event={"ID":"cf0c1e02-8817-4c08-a9c1-38fb654e6761","Type":"ContainerDied","Data":"7a8e29ba1faaf5cadd9918397c51eb0ff3ee01e445c63936c7581bb01b9f2352"} Oct 30 22:10:06.830742 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.830423 4615 scope.go:122] "RemoveContainer" containerID="71839e2963698832bbbb37a5eb2e2523a2c75b641ca6511a44ef748705cebbf5" Oct 30 22:10:06.839230 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.839184 4615 scope.go:122] "RemoveContainer" containerID="71839e2963698832bbbb37a5eb2e2523a2c75b641ca6511a44ef748705cebbf5" Oct 30 22:10:06.839433 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:10:06.839413 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71839e2963698832bbbb37a5eb2e2523a2c75b641ca6511a44ef748705cebbf5\": container with ID starting with 71839e2963698832bbbb37a5eb2e2523a2c75b641ca6511a44ef748705cebbf5 not found: ID does not exist" containerID="71839e2963698832bbbb37a5eb2e2523a2c75b641ca6511a44ef748705cebbf5" Oct 30 22:10:06.839471 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.839443 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71839e2963698832bbbb37a5eb2e2523a2c75b641ca6511a44ef748705cebbf5"} err="failed to get container status \"71839e2963698832bbbb37a5eb2e2523a2c75b641ca6511a44ef748705cebbf5\": rpc error: code = NotFound desc = could not find container \"71839e2963698832bbbb37a5eb2e2523a2c75b641ca6511a44ef748705cebbf5\": container with ID starting with 71839e2963698832bbbb37a5eb2e2523a2c75b641ca6511a44ef748705cebbf5 not found: ID does not exist" Oct 30 22:10:06.882335 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.882308 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68c99c9bdd-h582d"] Oct 30 22:10:06.894646 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:06.894613 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68c99c9bdd-h582d"] Oct 30 22:10:07.233137 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:07.233107 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf0c1e02-8817-4c08-a9c1-38fb654e6761" path="/var/lib/kubelet/pods/cf0c1e02-8817-4c08-a9c1-38fb654e6761/volumes" Oct 30 22:10:29.762954 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:10:29.762870 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:10:47.076028 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:47.075995 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:10:47.221032 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:47.220987 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:10:48.563143 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:10:48.563110 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:10:59.763352 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:10:59.763312 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:11:17.821686 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:17.821648 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:11:17.944904 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:17.944855 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:11:18.072337 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:18.072231 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:11:19.933383 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:19.933327 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:11:20.071707 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:20.071659 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:11:29.762447 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:11:29.762401 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:11:34.092736 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:34.092700 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:11:34.217719 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:34.217685 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:11:40.460110 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:40.460067 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:11:40.757185 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:40.757081 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:11:46.880874 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:46.880753 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:11:47.560638 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:47.560605 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:11:47.746377 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:47.746334 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:11:48.400530 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.400497 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr"] Oct 30 22:11:48.401027 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.400802 4615 cpu_manager.go:414] "containerMap: removing container" podUID="cf0c1e02-8817-4c08-a9c1-38fb654e6761" containerName="console" Oct 30 22:11:48.401027 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.400811 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf0c1e02-8817-4c08-a9c1-38fb654e6761" containerName="console" Oct 30 22:11:48.401027 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.400862 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf0c1e02-8817-4c08-a9c1-38fb654e6761" containerName="console" Oct 30 22:11:48.403798 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.403767 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr" Oct 30 22:11:48.429359 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.429329 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:11:48.432916 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.432881 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Oct 30 22:11:48.433150 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.433134 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" Oct 30 22:11:48.433244 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.433212 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Oct 30 22:11:48.433393 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.433372 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Oct 30 22:11:48.433393 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.433378 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-tj5cz\"" Oct 30 22:11:48.433695 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.433676 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Oct 30 22:11:48.446784 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.446752 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr"] Oct 30 22:11:48.499461 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.499423 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/4066d66d-9def-4863-bde2-6969dc490596-certificates\") pod \"keda-metrics-apiserver-86bd856cfb-4drzr\" (UID: \"4066d66d-9def-4863-bde2-6969dc490596\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr" Oct 30 22:11:48.499623 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.499476 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/4066d66d-9def-4863-bde2-6969dc490596-temp-vol\") pod \"keda-metrics-apiserver-86bd856cfb-4drzr\" (UID: \"4066d66d-9def-4863-bde2-6969dc490596\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr" Oct 30 22:11:48.499623 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.499560 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fkhz\" (UniqueName: \"kubernetes.io/projected/4066d66d-9def-4863-bde2-6969dc490596-kube-api-access-9fkhz\") pod \"keda-metrics-apiserver-86bd856cfb-4drzr\" (UID: \"4066d66d-9def-4863-bde2-6969dc490596\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr" Oct 30 22:11:48.600774 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.600728 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/4066d66d-9def-4863-bde2-6969dc490596-certificates\") pod \"keda-metrics-apiserver-86bd856cfb-4drzr\" (UID: \"4066d66d-9def-4863-bde2-6969dc490596\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr" Oct 30 22:11:48.600973 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.600796 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/4066d66d-9def-4863-bde2-6969dc490596-temp-vol\") pod \"keda-metrics-apiserver-86bd856cfb-4drzr\" (UID: \"4066d66d-9def-4863-bde2-6969dc490596\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr" Oct 30 22:11:48.600973 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.600839 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fkhz\" (UniqueName: \"kubernetes.io/projected/4066d66d-9def-4863-bde2-6969dc490596-kube-api-access-9fkhz\") pod \"keda-metrics-apiserver-86bd856cfb-4drzr\" (UID: \"4066d66d-9def-4863-bde2-6969dc490596\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr" Oct 30 22:11:48.601263 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.601238 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/4066d66d-9def-4863-bde2-6969dc490596-temp-vol\") pod \"keda-metrics-apiserver-86bd856cfb-4drzr\" (UID: \"4066d66d-9def-4863-bde2-6969dc490596\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr" Oct 30 22:11:48.603343 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.603318 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/4066d66d-9def-4863-bde2-6969dc490596-certificates\") pod \"keda-metrics-apiserver-86bd856cfb-4drzr\" (UID: \"4066d66d-9def-4863-bde2-6969dc490596\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr" Oct 30 22:11:48.625850 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.625816 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fkhz\" (UniqueName: \"kubernetes.io/projected/4066d66d-9def-4863-bde2-6969dc490596-kube-api-access-9fkhz\") pod \"keda-metrics-apiserver-86bd856cfb-4drzr\" (UID: \"4066d66d-9def-4863-bde2-6969dc490596\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr" Oct 30 22:11:48.628455 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.628430 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:11:48.716233 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.716152 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr" Oct 30 22:11:48.799078 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.798687 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-7b7c985c58-zgtxs"] Oct 30 22:11:48.802031 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.802009 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-7b7c985c58-zgtxs" Oct 30 22:11:48.819667 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.819646 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\"" Oct 30 22:11:48.840777 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.840746 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:11:48.852727 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.852696 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-7b7c985c58-zgtxs"] Oct 30 22:11:48.904799 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.904758 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8d66\" (UniqueName: \"kubernetes.io/projected/78260de1-476c-4c53-bb47-b80dceaf7cd7-kube-api-access-g8d66\") pod \"keda-admission-7b7c985c58-zgtxs\" (UID: \"78260de1-476c-4c53-bb47-b80dceaf7cd7\") " pod="openshift-keda/keda-admission-7b7c985c58-zgtxs" Oct 30 22:11:48.904985 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:48.904920 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/78260de1-476c-4c53-bb47-b80dceaf7cd7-certificates\") pod \"keda-admission-7b7c985c58-zgtxs\" (UID: \"78260de1-476c-4c53-bb47-b80dceaf7cd7\") " pod="openshift-keda/keda-admission-7b7c985c58-zgtxs" Oct 30 22:11:49.005910 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:49.005817 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/78260de1-476c-4c53-bb47-b80dceaf7cd7-certificates\") pod \"keda-admission-7b7c985c58-zgtxs\" (UID: \"78260de1-476c-4c53-bb47-b80dceaf7cd7\") " pod="openshift-keda/keda-admission-7b7c985c58-zgtxs" Oct 30 22:11:49.005910 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:49.005864 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8d66\" (UniqueName: \"kubernetes.io/projected/78260de1-476c-4c53-bb47-b80dceaf7cd7-kube-api-access-g8d66\") pod \"keda-admission-7b7c985c58-zgtxs\" (UID: \"78260de1-476c-4c53-bb47-b80dceaf7cd7\") " pod="openshift-keda/keda-admission-7b7c985c58-zgtxs" Oct 30 22:11:49.008548 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:49.008522 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/78260de1-476c-4c53-bb47-b80dceaf7cd7-certificates\") pod \"keda-admission-7b7c985c58-zgtxs\" (UID: \"78260de1-476c-4c53-bb47-b80dceaf7cd7\") " pod="openshift-keda/keda-admission-7b7c985c58-zgtxs" Oct 30 22:11:49.015664 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:49.015562 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr"] Oct 30 22:11:49.018962 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:11:49.018935 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4066d66d_9def_4863_bde2_6969dc490596.slice/crio-8db80800650143a46555e60049ddca53ef8ca666eef3a227027bc9440a21c2fc WatchSource:0}: Error finding container 8db80800650143a46555e60049ddca53ef8ca666eef3a227027bc9440a21c2fc: Status 404 returned error can't find the container with id 8db80800650143a46555e60049ddca53ef8ca666eef3a227027bc9440a21c2fc Oct 30 22:11:49.027567 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:49.027537 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:11:49.029238 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:49.029215 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8d66\" (UniqueName: \"kubernetes.io/projected/78260de1-476c-4c53-bb47-b80dceaf7cd7-kube-api-access-g8d66\") pod \"keda-admission-7b7c985c58-zgtxs\" (UID: \"78260de1-476c-4c53-bb47-b80dceaf7cd7\") " pod="openshift-keda/keda-admission-7b7c985c58-zgtxs" Oct 30 22:11:49.114777 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:49.114739 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-7b7c985c58-zgtxs" Oct 30 22:11:49.178101 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:49.178066 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr" event={"ID":"4066d66d-9def-4863-bde2-6969dc490596","Type":"ContainerStarted","Data":"8db80800650143a46555e60049ddca53ef8ca666eef3a227027bc9440a21c2fc"} Oct 30 22:11:49.408428 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:49.408389 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-7b7c985c58-zgtxs"] Oct 30 22:11:49.413216 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:11:49.412099 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78260de1_476c_4c53_bb47_b80dceaf7cd7.slice/crio-14564f95f85ab5eb20b32d69e6586b312ea0f517f5af361639e914b5a0000002 WatchSource:0}: Error finding container 14564f95f85ab5eb20b32d69e6586b312ea0f517f5af361639e914b5a0000002: Status 404 returned error can't find the container with id 14564f95f85ab5eb20b32d69e6586b312ea0f517f5af361639e914b5a0000002 Oct 30 22:11:49.616238 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:49.616208 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:11:49.924922 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:49.924882 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:11:50.144611 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:50.144576 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:11:50.185649 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:50.185555 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-7b7c985c58-zgtxs" event={"ID":"78260de1-476c-4c53-bb47-b80dceaf7cd7","Type":"ContainerStarted","Data":"14564f95f85ab5eb20b32d69e6586b312ea0f517f5af361639e914b5a0000002"} Oct 30 22:11:50.345846 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:50.345760 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:11:53.206726 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:53.206685 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr" event={"ID":"4066d66d-9def-4863-bde2-6969dc490596","Type":"ContainerStarted","Data":"1b45afadae3dcd448a9daee8aa89a1cd2f9c51c72d5c2d02b1dd32c94dbeb2fa"} Oct 30 22:11:53.207145 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:53.206827 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr" Oct 30 22:11:53.208224 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:53.208197 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-7b7c985c58-zgtxs" event={"ID":"78260de1-476c-4c53-bb47-b80dceaf7cd7","Type":"ContainerStarted","Data":"bf3ca1d605a4a96c3e55e7e866dbb043fa8142ee2617f2821ecd9b871af6c9dd"} Oct 30 22:11:53.208356 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:53.208344 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-7b7c985c58-zgtxs" Oct 30 22:11:53.246926 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:53.246882 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr" podStartSLOduration=1.5851652569999999 podStartE2EDuration="5.246871613s" totalImagesPullingTime="3.661706356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:53.246703086 +0000 UTC m=+230.162389399" watchObservedRunningTime="2026-10-30 22:11:53.246871613 +0000 UTC m=+230.162557910" Oct 30 22:11:53.288042 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:53.287992 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-keda/keda-admission-7b7c985c58-zgtxs" podStartSLOduration=2.297473319 podStartE2EDuration="5.287979059s" totalImagesPullingTime="2.99050574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:11:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:11:53.286579936 +0000 UTC m=+230.202266239" watchObservedRunningTime="2026-10-30 22:11:53.287979059 +0000 UTC m=+230.203665362" Oct 30 22:11:58.143171 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:58.143126 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:11:58.718384 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:58.718349 4615 patch_prober.go:29] interesting pod/keda-metrics-apiserver-86bd856cfb-4drzr container/keda-metrics-apiserver namespace/openshift-keda: Liveness probe status=failure output="Get \"https://10.134.0.22:6443/healthz\": dial tcp 10.134.0.22:6443: connect: connection refused" start-of-body= Oct 30 22:11:58.718538 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:11:58.718422 4615 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr" podUID="4066d66d-9def-4863-bde2-6969dc490596" containerName="keda-metrics-apiserver" probeResult="failure" output="Get \"https://10.134.0.22:6443/healthz\": dial tcp 10.134.0.22:6443: connect: connection refused" Oct 30 22:11:59.765741 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:11:59.763407 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:12:00.948616 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:12:00.948580 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:12:01.127128 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:12:01.127092 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:12:03.242984 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:12:03.242952 4615 generic.go:356] "Generic (PLEG): container finished" podID="4066d66d-9def-4863-bde2-6969dc490596" containerID="1b45afadae3dcd448a9daee8aa89a1cd2f9c51c72d5c2d02b1dd32c94dbeb2fa" exitCode=0 Oct 30 22:12:03.243394 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:12:03.243022 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr" event={"ID":"4066d66d-9def-4863-bde2-6969dc490596","Type":"ContainerDied","Data":"1b45afadae3dcd448a9daee8aa89a1cd2f9c51c72d5c2d02b1dd32c94dbeb2fa"} Oct 30 22:12:03.243394 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:12:03.243386 4615 scope.go:122] "RemoveContainer" containerID="1b45afadae3dcd448a9daee8aa89a1cd2f9c51c72d5c2d02b1dd32c94dbeb2fa" Oct 30 22:12:04.248760 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:12:04.248720 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr" event={"ID":"4066d66d-9def-4863-bde2-6969dc490596","Type":"ContainerStarted","Data":"159ed8c63e8e594ef17cf9044e4438895c3b87927e6b5d296b00b00526ca3916"} Oct 30 22:12:04.249190 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:12:04.248926 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr" Oct 30 22:12:14.214261 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:12:14.214226 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-7b7c985c58-zgtxs" Oct 30 22:12:15.257795 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:12:15.257767 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-4drzr" Oct 30 22:12:29.762888 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:12:29.762848 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:12:33.853988 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:12:33.853954 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:12:43.828407 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:12:43.828373 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:12:48.799182 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:12:48.799146 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:12:59.763235 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:12:59.763193 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:13:00.977249 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:00.977218 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:13:01.092700 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:01.092665 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:13:03.242401 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:03.242372 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:13:03.269896 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:03.269867 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:13:09.710934 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:09.710896 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:13:10.584789 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:10.584742 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:13:18.543291 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:18.543214 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:13:29.764418 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:13:29.764378 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:13:30.507681 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:30.507646 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:13:30.626012 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:30.625981 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:13:30.633781 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:30.633743 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-86c86ffb6b-k9tkz"] Oct 30 22:13:30.637058 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:30.637020 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-86c86ffb6b-k9tkz" Oct 30 22:13:30.656126 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:30.656093 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-86c86ffb6b-k9tkz"] Oct 30 22:13:30.725847 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:30.725816 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed7a2a5f-5f14-4a4e-ad01-942fdfebe949-cert\") pod \"llmisvc-controller-manager-86c86ffb6b-k9tkz\" (UID: \"ed7a2a5f-5f14-4a4e-ad01-942fdfebe949\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-k9tkz" Oct 30 22:13:30.725847 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:30.725841 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-xz5cn\"" Oct 30 22:13:30.726095 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:30.725868 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\"" Oct 30 22:13:30.726095 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:30.725890 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v2t5\" (UniqueName: \"kubernetes.io/projected/ed7a2a5f-5f14-4a4e-ad01-942fdfebe949-kube-api-access-8v2t5\") pod \"llmisvc-controller-manager-86c86ffb6b-k9tkz\" (UID: \"ed7a2a5f-5f14-4a4e-ad01-942fdfebe949\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-k9tkz" Oct 30 22:13:30.726399 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:30.726386 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Oct 30 22:13:30.726460 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:30.726445 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Oct 30 22:13:30.748983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:30.748943 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:13:30.827407 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:30.827323 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v2t5\" (UniqueName: \"kubernetes.io/projected/ed7a2a5f-5f14-4a4e-ad01-942fdfebe949-kube-api-access-8v2t5\") pod \"llmisvc-controller-manager-86c86ffb6b-k9tkz\" (UID: \"ed7a2a5f-5f14-4a4e-ad01-942fdfebe949\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-k9tkz" Oct 30 22:13:30.827407 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:30.827385 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed7a2a5f-5f14-4a4e-ad01-942fdfebe949-cert\") pod \"llmisvc-controller-manager-86c86ffb6b-k9tkz\" (UID: \"ed7a2a5f-5f14-4a4e-ad01-942fdfebe949\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-k9tkz" Oct 30 22:13:30.829928 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:30.829904 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed7a2a5f-5f14-4a4e-ad01-942fdfebe949-cert\") pod \"llmisvc-controller-manager-86c86ffb6b-k9tkz\" (UID: \"ed7a2a5f-5f14-4a4e-ad01-942fdfebe949\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-k9tkz" Oct 30 22:13:30.849707 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:30.849664 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v2t5\" (UniqueName: \"kubernetes.io/projected/ed7a2a5f-5f14-4a4e-ad01-942fdfebe949-kube-api-access-8v2t5\") pod \"llmisvc-controller-manager-86c86ffb6b-k9tkz\" (UID: \"ed7a2a5f-5f14-4a4e-ad01-942fdfebe949\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-k9tkz" Oct 30 22:13:30.864196 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:30.864166 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:13:30.949688 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:30.949653 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-86c86ffb6b-k9tkz" Oct 30 22:13:31.242778 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:31.242749 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-86c86ffb6b-k9tkz"] Oct 30 22:13:31.243994 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:13:31.243969 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7a2a5f_5f14_4a4e_ad01_942fdfebe949.slice/crio-52479e6eed157bd5c5ae421a3e606d128f48ee28f690959a21471f05722e32fa WatchSource:0}: Error finding container 52479e6eed157bd5c5ae421a3e606d128f48ee28f690959a21471f05722e32fa: Status 404 returned error can't find the container with id 52479e6eed157bd5c5ae421a3e606d128f48ee28f690959a21471f05722e32fa Oct 30 22:13:31.245471 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:31.245451 4615 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 22:13:31.558643 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:31.558557 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-86c86ffb6b-k9tkz" event={"ID":"ed7a2a5f-5f14-4a4e-ad01-942fdfebe949","Type":"ContainerStarted","Data":"52479e6eed157bd5c5ae421a3e606d128f48ee28f690959a21471f05722e32fa"} Oct 30 22:13:31.965439 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:31.965401 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:13:32.077080 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:32.077035 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:13:32.184860 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:32.184820 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:13:34.572184 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:34.572147 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-86c86ffb6b-k9tkz" event={"ID":"ed7a2a5f-5f14-4a4e-ad01-942fdfebe949","Type":"ContainerStarted","Data":"d244faee5812014592bb66dd0885efd4587f5c290784132e085ef0a452915049"} Oct 30 22:13:34.572590 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:34.572227 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-86c86ffb6b-k9tkz" Oct 30 22:13:34.611632 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:34.611571 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-86c86ffb6b-k9tkz" podStartSLOduration=2.157856568 podStartE2EDuration="4.611556599s" totalImagesPullingTime="2.453700031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:13:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:13:34.610416699 +0000 UTC m=+331.526103002" watchObservedRunningTime="2026-10-30 22:13:34.611556599 +0000 UTC m=+331.527242880" Oct 30 22:13:48.003832 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:48.003786 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:13:48.125725 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:48.125686 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:13:48.910274 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:13:48.910238 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:13:59.763609 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:13:59.763568 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:14:05.577503 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:14:05.577465 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-86c86ffb6b-k9tkz" Oct 30 22:14:29.763294 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:14:29.763256 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:14:49.022413 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:14:49.022336 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:14:51.854845 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:14:51.854743 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:14:51.987164 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:14:51.987093 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:14:52.105031 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:14:52.104929 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:14:52.253702 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:14:52.253672 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:14:56.084439 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:14:56.084400 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:14:56.198037 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:14:56.197987 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:14:59.762264 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:14:59.762225 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:15:08.003995 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:08.003958 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/s3-init-wjqck"] Oct 30 22:15:08.007347 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:08.007327 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-wjqck" Oct 30 22:15:08.022426 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:08.022407 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Oct 30 22:15:08.022512 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:08.022502 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-xw7xv\"" Oct 30 22:15:08.036109 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:08.036075 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-wjqck"] Oct 30 22:15:08.123029 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:08.122994 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64lhk\" (UniqueName: \"kubernetes.io/projected/d167ca51-c469-44f0-af6a-80dd449e9626-kube-api-access-64lhk\") pod \"s3-init-wjqck\" (UID: \"d167ca51-c469-44f0-af6a-80dd449e9626\") " pod="kserve/s3-init-wjqck" Oct 30 22:15:08.123214 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:08.123130 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d167ca51-c469-44f0-af6a-80dd449e9626-model-cache\") pod \"s3-init-wjqck\" (UID: \"d167ca51-c469-44f0-af6a-80dd449e9626\") " pod="kserve/s3-init-wjqck" Oct 30 22:15:08.123460 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:08.123441 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"hf-token\"" Oct 30 22:15:08.224208 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:08.224168 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d167ca51-c469-44f0-af6a-80dd449e9626-model-cache\") pod \"s3-init-wjqck\" (UID: \"d167ca51-c469-44f0-af6a-80dd449e9626\") " pod="kserve/s3-init-wjqck" Oct 30 22:15:08.224397 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:08.224218 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64lhk\" (UniqueName: \"kubernetes.io/projected/d167ca51-c469-44f0-af6a-80dd449e9626-kube-api-access-64lhk\") pod \"s3-init-wjqck\" (UID: \"d167ca51-c469-44f0-af6a-80dd449e9626\") " pod="kserve/s3-init-wjqck" Oct 30 22:15:08.224540 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:08.224516 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d167ca51-c469-44f0-af6a-80dd449e9626-model-cache\") pod \"s3-init-wjqck\" (UID: \"d167ca51-c469-44f0-af6a-80dd449e9626\") " pod="kserve/s3-init-wjqck" Oct 30 22:15:08.243929 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:08.243898 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64lhk\" (UniqueName: \"kubernetes.io/projected/d167ca51-c469-44f0-af6a-80dd449e9626-kube-api-access-64lhk\") pod \"s3-init-wjqck\" (UID: \"d167ca51-c469-44f0-af6a-80dd449e9626\") " pod="kserve/s3-init-wjqck" Oct 30 22:15:08.337125 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:08.337014 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-wjqck" Oct 30 22:15:08.627532 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:08.627496 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-wjqck"] Oct 30 22:15:08.629278 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:15:08.629241 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd167ca51_c469_44f0_af6a_80dd449e9626.slice/crio-137568c5a6ae3555cb7b0fe4ad0c00bc16475f658e99291cd209cf3f0f1f8981 WatchSource:0}: Error finding container 137568c5a6ae3555cb7b0fe4ad0c00bc16475f658e99291cd209cf3f0f1f8981: Status 404 returned error can't find the container with id 137568c5a6ae3555cb7b0fe4ad0c00bc16475f658e99291cd209cf3f0f1f8981 Oct 30 22:15:08.909983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:08.909948 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-wjqck" event={"ID":"d167ca51-c469-44f0-af6a-80dd449e9626","Type":"ContainerStarted","Data":"137568c5a6ae3555cb7b0fe4ad0c00bc16475f658e99291cd209cf3f0f1f8981"} Oct 30 22:15:11.924342 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:11.924310 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-wjqck" event={"ID":"d167ca51-c469-44f0-af6a-80dd449e9626","Type":"ContainerStarted","Data":"b465ea5c9cf3b936db5028d243bc4472874f4936b7661b961a0f0e2aad1e154e"} Oct 30 22:15:13.944165 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:13.944133 4615 generic.go:356] "Generic (PLEG): container finished" podID="d167ca51-c469-44f0-af6a-80dd449e9626" containerID="b465ea5c9cf3b936db5028d243bc4472874f4936b7661b961a0f0e2aad1e154e" exitCode=0 Oct 30 22:15:13.944558 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:13.944182 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-wjqck" event={"ID":"d167ca51-c469-44f0-af6a-80dd449e9626","Type":"ContainerDied","Data":"b465ea5c9cf3b936db5028d243bc4472874f4936b7661b961a0f0e2aad1e154e"} Oct 30 22:15:18.967411 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:18.967375 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-wjqck" event={"ID":"d167ca51-c469-44f0-af6a-80dd449e9626","Type":"ContainerStarted","Data":"931d31080cfe19776cc6b54962dff9591059fbc62673d31565e9c9cb481f0a9b"} Oct 30 22:15:19.011611 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:19.011557 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/s3-init-wjqck" podStartSLOduration=3.352475694 podStartE2EDuration="12.011542279s" totalImagesPullingTime="7.51821238s" totalInitContainerRuntime="1.140854205s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:15:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:15:19.010256068 +0000 UTC m=+435.925942369" watchObservedRunningTime="2026-10-30 22:15:19.011542279 +0000 UTC m=+435.927228580" Oct 30 22:15:29.762614 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:15:29.762577 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:15:48.899312 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:48.899275 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:15:49.010139 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:49.010104 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:15:49.130007 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:49.129963 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:15:49.244537 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:15:49.244452 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:15:59.761887 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:15:59.761850 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:16:01.619652 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:16:01.619619 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:16:11.887289 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:16:11.887249 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:16:24.205778 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:16:24.205742 4615 generic.go:356] "Generic (PLEG): container finished" podID="d167ca51-c469-44f0-af6a-80dd449e9626" containerID="931d31080cfe19776cc6b54962dff9591059fbc62673d31565e9c9cb481f0a9b" exitCode=0 Oct 30 22:16:24.206216 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:16:24.205808 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-wjqck" event={"ID":"d167ca51-c469-44f0-af6a-80dd449e9626","Type":"ContainerDied","Data":"931d31080cfe19776cc6b54962dff9591059fbc62673d31565e9c9cb481f0a9b"} Oct 30 22:16:25.334215 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:16:25.334186 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-wjqck" Oct 30 22:16:25.431327 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:16:25.431286 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d167ca51-c469-44f0-af6a-80dd449e9626-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d167ca51-c469-44f0-af6a-80dd449e9626-model-cache\") pod \"d167ca51-c469-44f0-af6a-80dd449e9626\" (UID: \"d167ca51-c469-44f0-af6a-80dd449e9626\") " Oct 30 22:16:25.431514 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:16:25.431349 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d167ca51-c469-44f0-af6a-80dd449e9626-kube-api-access-64lhk\" (UniqueName: \"kubernetes.io/projected/d167ca51-c469-44f0-af6a-80dd449e9626-kube-api-access-64lhk\") pod \"d167ca51-c469-44f0-af6a-80dd449e9626\" (UID: \"d167ca51-c469-44f0-af6a-80dd449e9626\") " Oct 30 22:16:25.433665 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:16:25.433623 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d167ca51-c469-44f0-af6a-80dd449e9626-kube-api-access-64lhk" pod "d167ca51-c469-44f0-af6a-80dd449e9626" (UID: "d167ca51-c469-44f0-af6a-80dd449e9626"). InnerVolumeSpecName "kube-api-access-64lhk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:16:25.449798 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:16:25.449756 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d167ca51-c469-44f0-af6a-80dd449e9626-model-cache" pod "d167ca51-c469-44f0-af6a-80dd449e9626" (UID: "d167ca51-c469-44f0-af6a-80dd449e9626"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:16:25.532200 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:16:25.532123 4615 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d167ca51-c469-44f0-af6a-80dd449e9626-model-cache\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:16:25.532200 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:16:25.532158 4615 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64lhk\" (UniqueName: \"kubernetes.io/projected/d167ca51-c469-44f0-af6a-80dd449e9626-kube-api-access-64lhk\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:16:26.214411 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:16:26.214380 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-wjqck" Oct 30 22:16:26.214411 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:16:26.214395 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-wjqck" event={"ID":"d167ca51-c469-44f0-af6a-80dd449e9626","Type":"ContainerDied","Data":"137568c5a6ae3555cb7b0fe4ad0c00bc16475f658e99291cd209cf3f0f1f8981"} Oct 30 22:16:26.214614 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:16:26.214432 4615 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="137568c5a6ae3555cb7b0fe4ad0c00bc16475f658e99291cd209cf3f0f1f8981" Oct 30 22:16:26.796364 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:16:26.796330 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:16:26.921006 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:16:26.920976 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:16:29.763105 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:16:29.763071 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:16:49.246720 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:16:49.246685 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:16:59.762819 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:16:59.762778 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:17:05.657788 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:05.657749 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:17:06.600558 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:06.600520 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:17:17.389248 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:17.389208 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:17:18.202516 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:18.202479 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:17:29.762245 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:17:29.762207 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:17:32.535028 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:32.534995 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:17:32.651450 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:32.651414 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:17:34.193840 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.193806 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:17:34.304860 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.304833 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:17:34.396597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.396566 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h"] Oct 30 22:17:34.396883 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.396870 4615 cpu_manager.go:414] "containerMap: removing container" podUID="d167ca51-c469-44f0-af6a-80dd449e9626" containerName="s3-init" Oct 30 22:17:34.396883 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.396884 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d167ca51-c469-44f0-af6a-80dd449e9626" containerName="s3-init" Oct 30 22:17:34.396988 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.396898 4615 cpu_manager.go:414] "containerMap: removing container" podUID="d167ca51-c469-44f0-af6a-80dd449e9626" containerName="download-hf-model" Oct 30 22:17:34.396988 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.396903 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d167ca51-c469-44f0-af6a-80dd449e9626" containerName="download-hf-model" Oct 30 22:17:34.396988 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.396967 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="d167ca51-c469-44f0-af6a-80dd449e9626" containerName="s3-init" Oct 30 22:17:34.399930 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.399914 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:17:34.415326 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.415305 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1b4518b221\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\"" Oct 30 22:17:34.427452 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.427422 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-1b4518b221\"/\"isvc-sklearn-graph-1-predictor-serving-cert\"" Oct 30 22:17:34.427452 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.427426 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-1b4518b221\"/\"seaweedfs-s3-creds\"" Oct 30 22:17:34.427584 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.427447 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1b4518b221\"/\"odh-kserve-custom-ca-bundle\"" Oct 30 22:17:34.427584 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.427479 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1b4518b221\"/\"openshift-service-ca.crt\"" Oct 30 22:17:34.427584 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.427485 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1b4518b221\"/\"kube-root-ca.crt\"" Oct 30 22:17:34.427584 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.427509 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-1b4518b221\"/\"default-dockercfg-zht6h\"" Oct 30 22:17:34.429212 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.429187 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h"] Oct 30 22:17:34.478865 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.478806 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqqtq\" (UniqueName: \"kubernetes.io/projected/bc6485ca-94f8-474c-a7a0-f185f67eebdc-kube-api-access-kqqtq\") pod \"isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h\" (UID: \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:17:34.478865 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.478843 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bc6485ca-94f8-474c-a7a0-f185f67eebdc-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h\" (UID: \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:17:34.478994 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.478919 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bc6485ca-94f8-474c-a7a0-f185f67eebdc-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h\" (UID: \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:17:34.478994 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.478954 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bc6485ca-94f8-474c-a7a0-f185f67eebdc-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h\" (UID: \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:17:34.478994 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.478975 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bc6485ca-94f8-474c-a7a0-f185f67eebdc-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h\" (UID: \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:17:34.579312 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.579288 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bc6485ca-94f8-474c-a7a0-f185f67eebdc-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h\" (UID: \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:17:34.579421 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.579329 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bc6485ca-94f8-474c-a7a0-f185f67eebdc-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h\" (UID: \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:17:34.579421 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.579357 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bc6485ca-94f8-474c-a7a0-f185f67eebdc-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h\" (UID: \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:17:34.579421 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.579384 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bc6485ca-94f8-474c-a7a0-f185f67eebdc-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h\" (UID: \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:17:34.579567 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.579450 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqqtq\" (UniqueName: \"kubernetes.io/projected/bc6485ca-94f8-474c-a7a0-f185f67eebdc-kube-api-access-kqqtq\") pod \"isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h\" (UID: \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:17:34.579862 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.579835 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bc6485ca-94f8-474c-a7a0-f185f67eebdc-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h\" (UID: \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:17:34.579981 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.579941 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bc6485ca-94f8-474c-a7a0-f185f67eebdc-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h\" (UID: \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:17:34.580151 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.580133 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bc6485ca-94f8-474c-a7a0-f185f67eebdc-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h\" (UID: \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:17:34.581721 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.581704 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bc6485ca-94f8-474c-a7a0-f185f67eebdc-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h\" (UID: \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:17:34.598678 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.598656 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqqtq\" (UniqueName: \"kubernetes.io/projected/bc6485ca-94f8-474c-a7a0-f185f67eebdc-kube-api-access-kqqtq\") pod \"isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h\" (UID: \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:17:34.701393 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.701352 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:17:34.716282 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.716255 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:17:34.848854 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:34.848826 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:17:35.018453 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.018424 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h"] Oct 30 22:17:35.099407 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.099382 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:17:35.213330 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.213292 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms"] Oct 30 22:17:35.217466 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.217445 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:17:35.235325 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.235220 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1b4518b221\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\"" Oct 30 22:17:35.235325 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.235305 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-1b4518b221\"/\"isvc-xgboost-graph-predictor-serving-cert\"" Oct 30 22:17:35.236584 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.236562 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms"] Oct 30 22:17:35.286582 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.286515 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cmhm\" (UniqueName: \"kubernetes.io/projected/c428d17c-304f-44af-9a55-c772082995a2-kube-api-access-9cmhm\") pod \"isvc-xgboost-graph-predictor-58fb998f8b-xvnms\" (UID: \"c428d17c-304f-44af-9a55-c772082995a2\") " pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:17:35.286582 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.286570 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c428d17c-304f-44af-9a55-c772082995a2-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-58fb998f8b-xvnms\" (UID: \"c428d17c-304f-44af-9a55-c772082995a2\") " pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:17:35.286772 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.286639 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c428d17c-304f-44af-9a55-c772082995a2-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-58fb998f8b-xvnms\" (UID: \"c428d17c-304f-44af-9a55-c772082995a2\") " pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:17:35.286772 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.286670 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c428d17c-304f-44af-9a55-c772082995a2-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-58fb998f8b-xvnms\" (UID: \"c428d17c-304f-44af-9a55-c772082995a2\") " pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:17:35.286772 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.286704 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c428d17c-304f-44af-9a55-c772082995a2-proxy-tls\") pod \"isvc-xgboost-graph-predictor-58fb998f8b-xvnms\" (UID: \"c428d17c-304f-44af-9a55-c772082995a2\") " pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:17:35.387691 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.387651 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c428d17c-304f-44af-9a55-c772082995a2-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-58fb998f8b-xvnms\" (UID: \"c428d17c-304f-44af-9a55-c772082995a2\") " pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:17:35.387883 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.387703 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c428d17c-304f-44af-9a55-c772082995a2-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-58fb998f8b-xvnms\" (UID: \"c428d17c-304f-44af-9a55-c772082995a2\") " pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:17:35.387883 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.387725 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c428d17c-304f-44af-9a55-c772082995a2-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-58fb998f8b-xvnms\" (UID: \"c428d17c-304f-44af-9a55-c772082995a2\") " pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:17:35.387883 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.387754 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c428d17c-304f-44af-9a55-c772082995a2-proxy-tls\") pod \"isvc-xgboost-graph-predictor-58fb998f8b-xvnms\" (UID: \"c428d17c-304f-44af-9a55-c772082995a2\") " pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:17:35.387883 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.387818 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cmhm\" (UniqueName: \"kubernetes.io/projected/c428d17c-304f-44af-9a55-c772082995a2-kube-api-access-9cmhm\") pod \"isvc-xgboost-graph-predictor-58fb998f8b-xvnms\" (UID: \"c428d17c-304f-44af-9a55-c772082995a2\") " pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:17:35.388160 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.388138 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c428d17c-304f-44af-9a55-c772082995a2-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-58fb998f8b-xvnms\" (UID: \"c428d17c-304f-44af-9a55-c772082995a2\") " pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:17:35.388354 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.388332 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c428d17c-304f-44af-9a55-c772082995a2-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-58fb998f8b-xvnms\" (UID: \"c428d17c-304f-44af-9a55-c772082995a2\") " pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:17:35.388422 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.388354 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c428d17c-304f-44af-9a55-c772082995a2-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-58fb998f8b-xvnms\" (UID: \"c428d17c-304f-44af-9a55-c772082995a2\") " pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:17:35.390073 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.390039 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c428d17c-304f-44af-9a55-c772082995a2-proxy-tls\") pod \"isvc-xgboost-graph-predictor-58fb998f8b-xvnms\" (UID: \"c428d17c-304f-44af-9a55-c772082995a2\") " pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:17:35.407921 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.407902 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cmhm\" (UniqueName: \"kubernetes.io/projected/c428d17c-304f-44af-9a55-c772082995a2-kube-api-access-9cmhm\") pod \"isvc-xgboost-graph-predictor-58fb998f8b-xvnms\" (UID: \"c428d17c-304f-44af-9a55-c772082995a2\") " pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:17:35.469334 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.469294 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" event={"ID":"bc6485ca-94f8-474c-a7a0-f185f67eebdc","Type":"ContainerStarted","Data":"688452f551fbd3a080057645b984502837c957f9be1d86b4d37c406131c3b18c"} Oct 30 22:17:35.469484 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.469405 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" event={"ID":"bc6485ca-94f8-474c-a7a0-f185f67eebdc","Type":"ContainerStarted","Data":"a4d14d3c4123f8a176f7b8c83302255f0d7d11e25bb744f71657c6740426c620"} Oct 30 22:17:35.536001 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.535968 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:17:35.807176 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.807097 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln"] Oct 30 22:17:35.811500 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.811476 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" Oct 30 22:17:35.826504 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.826482 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-cc765036c8\"/\"error-404-isvc-b9b03-predictor-serving-cert\"" Oct 30 22:17:35.826504 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.826497 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-cc765036c8\"/\"error-404-isvc-b9b03-kube-rbac-proxy-sar-config\"" Oct 30 22:17:35.826680 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.826620 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-cc765036c8\"/\"kube-root-ca.crt\"" Oct 30 22:17:35.826680 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.826639 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-cc765036c8\"/\"default-dockercfg-r6qmt\"" Oct 30 22:17:35.829220 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.829204 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-cc765036c8\"/\"openshift-service-ca.crt\"" Oct 30 22:17:35.831344 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.831326 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln"] Oct 30 22:17:35.851615 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.851592 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms"] Oct 30 22:17:35.896903 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.896881 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f52379ef-2e4a-4221-b2ce-fa1e82bdd593-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-b9b03-predictor-7f789594bd-xvlln\" (UID: \"f52379ef-2e4a-4221-b2ce-fa1e82bdd593\") " pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" Oct 30 22:17:35.897001 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.896933 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82smv\" (UniqueName: \"kubernetes.io/projected/f52379ef-2e4a-4221-b2ce-fa1e82bdd593-kube-api-access-82smv\") pod \"error-404-isvc-b9b03-predictor-7f789594bd-xvlln\" (UID: \"f52379ef-2e4a-4221-b2ce-fa1e82bdd593\") " pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" Oct 30 22:17:35.897001 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.896957 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f52379ef-2e4a-4221-b2ce-fa1e82bdd593-proxy-tls\") pod \"error-404-isvc-b9b03-predictor-7f789594bd-xvlln\" (UID: \"f52379ef-2e4a-4221-b2ce-fa1e82bdd593\") " pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" Oct 30 22:17:35.997594 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.997564 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f52379ef-2e4a-4221-b2ce-fa1e82bdd593-proxy-tls\") pod \"error-404-isvc-b9b03-predictor-7f789594bd-xvlln\" (UID: \"f52379ef-2e4a-4221-b2ce-fa1e82bdd593\") " pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" Oct 30 22:17:35.997735 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.997610 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f52379ef-2e4a-4221-b2ce-fa1e82bdd593-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-b9b03-predictor-7f789594bd-xvlln\" (UID: \"f52379ef-2e4a-4221-b2ce-fa1e82bdd593\") " pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" Oct 30 22:17:35.997735 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.997658 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82smv\" (UniqueName: \"kubernetes.io/projected/f52379ef-2e4a-4221-b2ce-fa1e82bdd593-kube-api-access-82smv\") pod \"error-404-isvc-b9b03-predictor-7f789594bd-xvlln\" (UID: \"f52379ef-2e4a-4221-b2ce-fa1e82bdd593\") " pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" Oct 30 22:17:35.998317 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.998294 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f52379ef-2e4a-4221-b2ce-fa1e82bdd593-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-b9b03-predictor-7f789594bd-xvlln\" (UID: \"f52379ef-2e4a-4221-b2ce-fa1e82bdd593\") " pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" Oct 30 22:17:35.999878 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:35.999852 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f52379ef-2e4a-4221-b2ce-fa1e82bdd593-proxy-tls\") pod \"error-404-isvc-b9b03-predictor-7f789594bd-xvlln\" (UID: \"f52379ef-2e4a-4221-b2ce-fa1e82bdd593\") " pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" Oct 30 22:17:36.017118 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.017093 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82smv\" (UniqueName: \"kubernetes.io/projected/f52379ef-2e4a-4221-b2ce-fa1e82bdd593-kube-api-access-82smv\") pod \"error-404-isvc-b9b03-predictor-7f789594bd-xvlln\" (UID: \"f52379ef-2e4a-4221-b2ce-fa1e82bdd593\") " pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" Oct 30 22:17:36.126709 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.126631 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" Oct 30 22:17:36.192869 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.192827 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:17:36.297496 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.297461 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx"] Oct 30 22:17:36.301335 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.301315 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:17:36.320596 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.320372 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1b4518b221\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\"" Oct 30 22:17:36.320596 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.320550 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-1b4518b221\"/\"isvc-sklearn-graph-2-predictor-serving-cert\"" Oct 30 22:17:36.321983 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.321960 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx"] Oct 30 22:17:36.335405 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.335379 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:17:36.401089 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.401067 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e27299f9-1e59-4c12-8baa-70f3d276f404-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-79577dc58b-djdnx\" (UID: \"e27299f9-1e59-4c12-8baa-70f3d276f404\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:17:36.401229 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.401100 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5zwf\" (UniqueName: \"kubernetes.io/projected/e27299f9-1e59-4c12-8baa-70f3d276f404-kube-api-access-q5zwf\") pod \"isvc-sklearn-graph-2-predictor-79577dc58b-djdnx\" (UID: \"e27299f9-1e59-4c12-8baa-70f3d276f404\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:17:36.401229 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.401135 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e27299f9-1e59-4c12-8baa-70f3d276f404-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-79577dc58b-djdnx\" (UID: \"e27299f9-1e59-4c12-8baa-70f3d276f404\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:17:36.401229 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.401201 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e27299f9-1e59-4c12-8baa-70f3d276f404-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-79577dc58b-djdnx\" (UID: \"e27299f9-1e59-4c12-8baa-70f3d276f404\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:17:36.401333 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.401241 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e27299f9-1e59-4c12-8baa-70f3d276f404-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-79577dc58b-djdnx\" (UID: \"e27299f9-1e59-4c12-8baa-70f3d276f404\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:17:36.419223 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.419200 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln"] Oct 30 22:17:36.420976 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:17:36.420953 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf52379ef_2e4a_4221_b2ce_fa1e82bdd593.slice/crio-33c17b0d813e44469732b8e5c630ac66a743e56ef1536d285b56836f7ee96a2d WatchSource:0}: Error finding container 33c17b0d813e44469732b8e5c630ac66a743e56ef1536d285b56836f7ee96a2d: Status 404 returned error can't find the container with id 33c17b0d813e44469732b8e5c630ac66a743e56ef1536d285b56836f7ee96a2d Oct 30 22:17:36.474240 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.474210 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" event={"ID":"f52379ef-2e4a-4221-b2ce-fa1e82bdd593","Type":"ContainerStarted","Data":"33c17b0d813e44469732b8e5c630ac66a743e56ef1536d285b56836f7ee96a2d"} Oct 30 22:17:36.475760 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.475737 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" event={"ID":"c428d17c-304f-44af-9a55-c772082995a2","Type":"ContainerStarted","Data":"1972b01f1fff6e076c09ba460da8a22df13c26fc58813a4c66571c15cc84659f"} Oct 30 22:17:36.475854 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.475788 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" event={"ID":"c428d17c-304f-44af-9a55-c772082995a2","Type":"ContainerStarted","Data":"f7f98c62c8e17dd4e26e11703b9e657be62841f76c59b44dc8a70d5d370c3d46"} Oct 30 22:17:36.502022 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.501995 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e27299f9-1e59-4c12-8baa-70f3d276f404-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-79577dc58b-djdnx\" (UID: \"e27299f9-1e59-4c12-8baa-70f3d276f404\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:17:36.502143 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.502034 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5zwf\" (UniqueName: \"kubernetes.io/projected/e27299f9-1e59-4c12-8baa-70f3d276f404-kube-api-access-q5zwf\") pod \"isvc-sklearn-graph-2-predictor-79577dc58b-djdnx\" (UID: \"e27299f9-1e59-4c12-8baa-70f3d276f404\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:17:36.502143 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.502114 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e27299f9-1e59-4c12-8baa-70f3d276f404-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-79577dc58b-djdnx\" (UID: \"e27299f9-1e59-4c12-8baa-70f3d276f404\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:17:36.502258 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.502150 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e27299f9-1e59-4c12-8baa-70f3d276f404-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-79577dc58b-djdnx\" (UID: \"e27299f9-1e59-4c12-8baa-70f3d276f404\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:17:36.502258 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.502211 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e27299f9-1e59-4c12-8baa-70f3d276f404-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-79577dc58b-djdnx\" (UID: \"e27299f9-1e59-4c12-8baa-70f3d276f404\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:17:36.502689 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.502661 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e27299f9-1e59-4c12-8baa-70f3d276f404-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-79577dc58b-djdnx\" (UID: \"e27299f9-1e59-4c12-8baa-70f3d276f404\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:17:36.502931 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.502915 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e27299f9-1e59-4c12-8baa-70f3d276f404-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-79577dc58b-djdnx\" (UID: \"e27299f9-1e59-4c12-8baa-70f3d276f404\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:17:36.503118 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.503096 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e27299f9-1e59-4c12-8baa-70f3d276f404-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-79577dc58b-djdnx\" (UID: \"e27299f9-1e59-4c12-8baa-70f3d276f404\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:17:36.504403 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.504384 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e27299f9-1e59-4c12-8baa-70f3d276f404-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-79577dc58b-djdnx\" (UID: \"e27299f9-1e59-4c12-8baa-70f3d276f404\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:17:36.521807 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.521776 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5zwf\" (UniqueName: \"kubernetes.io/projected/e27299f9-1e59-4c12-8baa-70f3d276f404-kube-api-access-q5zwf\") pod \"isvc-sklearn-graph-2-predictor-79577dc58b-djdnx\" (UID: \"e27299f9-1e59-4c12-8baa-70f3d276f404\") " pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:17:36.619925 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.619894 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:17:36.922551 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:36.922518 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx"] Oct 30 22:17:36.926398 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:17:36.926366 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode27299f9_1e59_4c12_8baa_70f3d276f404.slice/crio-47cb0ca8b136a06514b4dfeb737b10dba53882f429a440ae4c30c6a2b490b257 WatchSource:0}: Error finding container 47cb0ca8b136a06514b4dfeb737b10dba53882f429a440ae4c30c6a2b490b257: Status 404 returned error can't find the container with id 47cb0ca8b136a06514b4dfeb737b10dba53882f429a440ae4c30c6a2b490b257 Oct 30 22:17:37.492963 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:37.492925 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" event={"ID":"e27299f9-1e59-4c12-8baa-70f3d276f404","Type":"ContainerStarted","Data":"d777caa009f7fde17a60bd674364ff4f247fb5f7bfff2eb871f116dd3589c994"} Oct 30 22:17:37.493449 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:37.492994 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" event={"ID":"e27299f9-1e59-4c12-8baa-70f3d276f404","Type":"ContainerStarted","Data":"47cb0ca8b136a06514b4dfeb737b10dba53882f429a440ae4c30c6a2b490b257"} Oct 30 22:17:39.506552 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:39.506512 4615 generic.go:356] "Generic (PLEG): container finished" podID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerID="688452f551fbd3a080057645b984502837c957f9be1d86b4d37c406131c3b18c" exitCode=0 Oct 30 22:17:39.507003 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:39.506565 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" event={"ID":"bc6485ca-94f8-474c-a7a0-f185f67eebdc","Type":"ContainerDied","Data":"688452f551fbd3a080057645b984502837c957f9be1d86b4d37c406131c3b18c"} Oct 30 22:17:40.515690 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:40.515651 4615 generic.go:356] "Generic (PLEG): container finished" podID="c428d17c-304f-44af-9a55-c772082995a2" containerID="1972b01f1fff6e076c09ba460da8a22df13c26fc58813a4c66571c15cc84659f" exitCode=0 Oct 30 22:17:40.516188 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:40.515748 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" event={"ID":"c428d17c-304f-44af-9a55-c772082995a2","Type":"ContainerDied","Data":"1972b01f1fff6e076c09ba460da8a22df13c26fc58813a4c66571c15cc84659f"} Oct 30 22:17:41.532933 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:41.532894 4615 generic.go:356] "Generic (PLEG): container finished" podID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerID="d777caa009f7fde17a60bd674364ff4f247fb5f7bfff2eb871f116dd3589c994" exitCode=0 Oct 30 22:17:41.533402 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:41.533012 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" event={"ID":"e27299f9-1e59-4c12-8baa-70f3d276f404","Type":"ContainerDied","Data":"d777caa009f7fde17a60bd674364ff4f247fb5f7bfff2eb871f116dd3589c994"} Oct 30 22:17:49.362985 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:17:49.362727 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:17:59.774811 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:17:59.774766 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:18:01.696132 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:01.696092 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" event={"ID":"f52379ef-2e4a-4221-b2ce-fa1e82bdd593","Type":"ContainerStarted","Data":"27e9c904a3274923a966fa6316e52677df79b2faf7350bdc443faf8beb397b8d"} Oct 30 22:18:02.704295 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:02.704254 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" event={"ID":"bc6485ca-94f8-474c-a7a0-f185f67eebdc","Type":"ContainerStarted","Data":"e6db15913bd808cd57d4cc0467502eb158570393423cd10b7e68838a01c23197"} Oct 30 22:18:03.291583 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:03.291530 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:18:03.323175 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:03.323143 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:18:04.718199 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:04.718163 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" event={"ID":"bc6485ca-94f8-474c-a7a0-f185f67eebdc","Type":"ContainerStarted","Data":"c376c3cdc8c28471d8577f3a3fe73a9262cf14541a94acb36f9b53d85f57038d"} Oct 30 22:18:04.724064 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:04.722129 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" event={"ID":"f52379ef-2e4a-4221-b2ce-fa1e82bdd593","Type":"ContainerStarted","Data":"0e535b938028e111e194362360075460ca97570ff380e1ae3506b8e13ba628db"} Oct 30 22:18:05.729393 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:05.729352 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" event={"ID":"e27299f9-1e59-4c12-8baa-70f3d276f404","Type":"ContainerStarted","Data":"076b1fbaea312dbad83b2b89a99b6c0ac775fd3a89da8199fc32d92f2a2c4365"} Oct 30 22:18:05.729942 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:05.729428 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" event={"ID":"e27299f9-1e59-4c12-8baa-70f3d276f404","Type":"ContainerStarted","Data":"dcd520aa767fcc521a67aa62d0c8332860060008a51f429008c1d85a9f6039c2"} Oct 30 22:18:05.729942 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:05.729655 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:18:05.729942 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:05.729875 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:18:05.730237 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:05.730214 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:18:05.730323 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:05.730269 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:18:05.730945 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:05.730915 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Oct 30 22:18:05.731448 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:05.731423 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Oct 30 22:18:05.787359 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:05.787312 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" podStartSLOduration=2.818262239 podStartE2EDuration="31.787296127s" totalImagesPullingTime="24.972724946s" totalInitContainerRuntime="3.996308942s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:05.785721552 +0000 UTC m=+602.701407852" watchObservedRunningTime="2026-10-30 22:18:05.787296127 +0000 UTC m=+602.702982432" Oct 30 22:18:05.830482 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:05.830418 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" podStartSLOduration=2.117656826 podStartE2EDuration="29.830398888s" totalImagesPullingTime="23.654426628s" totalInitContainerRuntime="4.058315434s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:05.828929877 +0000 UTC m=+602.744616185" watchObservedRunningTime="2026-10-30 22:18:05.830398888 +0000 UTC m=+602.746085192" Oct 30 22:18:05.872584 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:05.872114 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" podStartSLOduration=3.015467834 podStartE2EDuration="30.872094513s" totalImagesPullingTime="27.856626679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:05.871666618 +0000 UTC m=+602.787352921" watchObservedRunningTime="2026-10-30 22:18:05.872094513 +0000 UTC m=+602.787780820" Oct 30 22:18:06.736109 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:06.735267 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" Oct 30 22:18:06.736109 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:06.735322 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Oct 30 22:18:06.736109 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:06.736097 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" Oct 30 22:18:06.739035 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:06.736720 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Oct 30 22:18:06.739035 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:06.737228 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" podUID="f52379ef-2e4a-4221-b2ce-fa1e82bdd593" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Oct 30 22:18:07.739111 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:07.739065 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" podUID="f52379ef-2e4a-4221-b2ce-fa1e82bdd593" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Oct 30 22:18:08.743314 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:08.743280 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" podUID="f52379ef-2e4a-4221-b2ce-fa1e82bdd593" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Oct 30 22:18:10.753532 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:10.753498 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" event={"ID":"c428d17c-304f-44af-9a55-c772082995a2","Type":"ContainerStarted","Data":"be8058ad59e80c000e18d2fb98cddec64089db24822149358a544a6afec0c0fc"} Oct 30 22:18:10.753965 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:10.753558 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" event={"ID":"c428d17c-304f-44af-9a55-c772082995a2","Type":"ContainerStarted","Data":"78644f4e4325a39481b24c00c1016cc5ced918cdfbfebb3343b830f6d63e597c"} Oct 30 22:18:10.753965 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:10.753753 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:18:11.739556 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:11.739522 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:18:11.739940 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:11.739919 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:18:11.740073 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:11.739940 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Oct 30 22:18:11.740442 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:11.740419 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Oct 30 22:18:11.757790 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:11.757763 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:18:11.758863 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:11.758839 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" podUID="c428d17c-304f-44af-9a55-c772082995a2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Oct 30 22:18:11.796903 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:11.796857 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" podStartSLOduration=3.558648883 podStartE2EDuration="36.796845906s" totalImagesPullingTime="29.249874327s" totalInitContainerRuntime="3.988322696s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:17:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:10.808348425 +0000 UTC m=+607.724034727" watchObservedRunningTime="2026-10-30 22:18:11.796845906 +0000 UTC m=+608.712532207" Oct 30 22:18:12.761982 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:12.761940 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" podUID="c428d17c-304f-44af-9a55-c772082995a2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Oct 30 22:18:13.747237 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:13.747209 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" Oct 30 22:18:13.747623 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:13.747600 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" podUID="f52379ef-2e4a-4221-b2ce-fa1e82bdd593" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Oct 30 22:18:17.766852 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:17.766821 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:18:17.767409 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:17.767383 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" podUID="c428d17c-304f-44af-9a55-c772082995a2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Oct 30 22:18:21.740271 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:21.740226 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Oct 30 22:18:21.740699 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:21.740382 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Oct 30 22:18:23.747984 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:23.747947 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" podUID="f52379ef-2e4a-4221-b2ce-fa1e82bdd593" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Oct 30 22:18:27.767855 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:27.767819 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" podUID="c428d17c-304f-44af-9a55-c772082995a2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Oct 30 22:18:29.762839 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:18:29.762800 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:18:31.740265 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:31.740223 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Oct 30 22:18:31.740683 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:31.740343 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Oct 30 22:18:33.747669 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:33.747622 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" podUID="f52379ef-2e4a-4221-b2ce-fa1e82bdd593" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Oct 30 22:18:37.768090 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:37.768023 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" podUID="c428d17c-304f-44af-9a55-c772082995a2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Oct 30 22:18:41.740316 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:41.740278 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Oct 30 22:18:41.740724 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:41.740415 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Oct 30 22:18:43.748299 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:43.748262 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" podUID="f52379ef-2e4a-4221-b2ce-fa1e82bdd593" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Oct 30 22:18:47.767724 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:47.767679 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" podUID="c428d17c-304f-44af-9a55-c772082995a2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Oct 30 22:18:49.476173 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:49.476115 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:18:51.740183 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:51.740150 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Oct 30 22:18:51.740602 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:51.740461 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Oct 30 22:18:53.748379 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:53.748336 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" podUID="f52379ef-2e4a-4221-b2ce-fa1e82bdd593" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Oct 30 22:18:57.768137 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:18:57.768096 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" podUID="c428d17c-304f-44af-9a55-c772082995a2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Oct 30 22:18:59.763610 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:18:59.763575 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:19:01.740497 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:01.740453 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Oct 30 22:19:01.740988 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:01.740454 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Oct 30 22:19:03.748229 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:03.748194 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" Oct 30 22:19:07.768114 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:07.768072 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" podUID="c428d17c-304f-44af-9a55-c772082995a2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Oct 30 22:19:11.740066 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:11.740007 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Oct 30 22:19:11.740511 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:11.740438 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Oct 30 22:19:14.456665 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:14.456621 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:19:14.573613 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:14.573578 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:19:17.768022 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:17.767934 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" podUID="c428d17c-304f-44af-9a55-c772082995a2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Oct 30 22:19:18.229028 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:18.228980 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" podUID="c428d17c-304f-44af-9a55-c772082995a2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Oct 30 22:19:19.229095 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:19.229023 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Oct 30 22:19:21.740086 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:21.740020 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused" Oct 30 22:19:25.732780 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:25.732746 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln"] Oct 30 22:19:25.733276 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:25.733103 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" podUID="f52379ef-2e4a-4221-b2ce-fa1e82bdd593" containerName="kserve-container" containerID="cri-o://27e9c904a3274923a966fa6316e52677df79b2faf7350bdc443faf8beb397b8d" gracePeriod=30 Oct 30 22:19:25.733276 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:25.733109 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" podUID="f52379ef-2e4a-4221-b2ce-fa1e82bdd593" containerName="kube-rbac-proxy" containerID="cri-o://0e535b938028e111e194362360075460ca97570ff380e1ae3506b8e13ba628db" gracePeriod=30 Oct 30 22:19:25.843125 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:25.843085 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:19:25.960309 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:25.960274 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:19:26.077082 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:26.076968 4615 generic.go:356] "Generic (PLEG): container finished" podID="f52379ef-2e4a-4221-b2ce-fa1e82bdd593" containerID="0e535b938028e111e194362360075460ca97570ff380e1ae3506b8e13ba628db" exitCode=0 Oct 30 22:19:26.077082 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:26.077058 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" event={"ID":"f52379ef-2e4a-4221-b2ce-fa1e82bdd593","Type":"ContainerDied","Data":"0e535b938028e111e194362360075460ca97570ff380e1ae3506b8e13ba628db"} Oct 30 22:19:28.229920 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:28.229879 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:19:28.744398 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:28.744360 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" podUID="f52379ef-2e4a-4221-b2ce-fa1e82bdd593" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.28:8643/healthz\": dial tcp 10.134.0.28:8643: connect: connection refused" Oct 30 22:19:29.233978 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:29.233942 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:19:29.477115 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:29.477089 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" Oct 30 22:19:29.537898 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:29.537825 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f52379ef-2e4a-4221-b2ce-fa1e82bdd593-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f52379ef-2e4a-4221-b2ce-fa1e82bdd593-kube-rbac-proxy-sar-config\") pod \"f52379ef-2e4a-4221-b2ce-fa1e82bdd593\" (UID: \"f52379ef-2e4a-4221-b2ce-fa1e82bdd593\") " Oct 30 22:19:29.537898 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:29.537887 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f52379ef-2e4a-4221-b2ce-fa1e82bdd593-kube-api-access-82smv\" (UniqueName: \"kubernetes.io/projected/f52379ef-2e4a-4221-b2ce-fa1e82bdd593-kube-api-access-82smv\") pod \"f52379ef-2e4a-4221-b2ce-fa1e82bdd593\" (UID: \"f52379ef-2e4a-4221-b2ce-fa1e82bdd593\") " Oct 30 22:19:29.538153 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:29.537928 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f52379ef-2e4a-4221-b2ce-fa1e82bdd593-proxy-tls\" (UniqueName: \"kubernetes.io/secret/f52379ef-2e4a-4221-b2ce-fa1e82bdd593-proxy-tls\") pod \"f52379ef-2e4a-4221-b2ce-fa1e82bdd593\" (UID: \"f52379ef-2e4a-4221-b2ce-fa1e82bdd593\") " Oct 30 22:19:29.538229 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:29.538205 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f52379ef-2e4a-4221-b2ce-fa1e82bdd593-kube-rbac-proxy-sar-config" pod "f52379ef-2e4a-4221-b2ce-fa1e82bdd593" (UID: "f52379ef-2e4a-4221-b2ce-fa1e82bdd593"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:19:29.540136 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:29.540103 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f52379ef-2e4a-4221-b2ce-fa1e82bdd593-proxy-tls" pod "f52379ef-2e4a-4221-b2ce-fa1e82bdd593" (UID: "f52379ef-2e4a-4221-b2ce-fa1e82bdd593"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:19:29.540238 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:29.540135 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f52379ef-2e4a-4221-b2ce-fa1e82bdd593-kube-api-access-82smv" pod "f52379ef-2e4a-4221-b2ce-fa1e82bdd593" (UID: "f52379ef-2e4a-4221-b2ce-fa1e82bdd593"). InnerVolumeSpecName "kube-api-access-82smv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:19:29.638985 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:29.638946 4615 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f52379ef-2e4a-4221-b2ce-fa1e82bdd593-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:19:29.638985 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:29.638977 4615 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82smv\" (UniqueName: \"kubernetes.io/projected/f52379ef-2e4a-4221-b2ce-fa1e82bdd593-kube-api-access-82smv\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:19:29.638985 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:29.638986 4615 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f52379ef-2e4a-4221-b2ce-fa1e82bdd593-proxy-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:19:29.762369 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:19:29.762330 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:19:30.094731 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:30.094682 4615 generic.go:356] "Generic (PLEG): container finished" podID="f52379ef-2e4a-4221-b2ce-fa1e82bdd593" containerID="27e9c904a3274923a966fa6316e52677df79b2faf7350bdc443faf8beb397b8d" exitCode=0 Oct 30 22:19:30.094907 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:30.094746 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" Oct 30 22:19:30.094907 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:30.094762 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" event={"ID":"f52379ef-2e4a-4221-b2ce-fa1e82bdd593","Type":"ContainerDied","Data":"27e9c904a3274923a966fa6316e52677df79b2faf7350bdc443faf8beb397b8d"} Oct 30 22:19:30.094907 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:30.094822 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln" event={"ID":"f52379ef-2e4a-4221-b2ce-fa1e82bdd593","Type":"ContainerDied","Data":"33c17b0d813e44469732b8e5c630ac66a743e56ef1536d285b56836f7ee96a2d"} Oct 30 22:19:30.094907 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:30.094850 4615 scope.go:122] "RemoveContainer" containerID="0e535b938028e111e194362360075460ca97570ff380e1ae3506b8e13ba628db" Oct 30 22:19:30.103247 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:30.103228 4615 scope.go:122] "RemoveContainer" containerID="27e9c904a3274923a966fa6316e52677df79b2faf7350bdc443faf8beb397b8d" Oct 30 22:19:30.110565 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:30.110545 4615 scope.go:122] "RemoveContainer" containerID="0e535b938028e111e194362360075460ca97570ff380e1ae3506b8e13ba628db" Oct 30 22:19:30.110824 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:19:30.110808 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e535b938028e111e194362360075460ca97570ff380e1ae3506b8e13ba628db\": container with ID starting with 0e535b938028e111e194362360075460ca97570ff380e1ae3506b8e13ba628db not found: ID does not exist" containerID="0e535b938028e111e194362360075460ca97570ff380e1ae3506b8e13ba628db" Oct 30 22:19:30.110862 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:30.110847 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e535b938028e111e194362360075460ca97570ff380e1ae3506b8e13ba628db"} err="failed to get container status \"0e535b938028e111e194362360075460ca97570ff380e1ae3506b8e13ba628db\": rpc error: code = NotFound desc = could not find container \"0e535b938028e111e194362360075460ca97570ff380e1ae3506b8e13ba628db\": container with ID starting with 0e535b938028e111e194362360075460ca97570ff380e1ae3506b8e13ba628db not found: ID does not exist" Oct 30 22:19:30.110899 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:30.110864 4615 scope.go:122] "RemoveContainer" containerID="27e9c904a3274923a966fa6316e52677df79b2faf7350bdc443faf8beb397b8d" Oct 30 22:19:30.111105 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:19:30.111085 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27e9c904a3274923a966fa6316e52677df79b2faf7350bdc443faf8beb397b8d\": container with ID starting with 27e9c904a3274923a966fa6316e52677df79b2faf7350bdc443faf8beb397b8d not found: ID does not exist" containerID="27e9c904a3274923a966fa6316e52677df79b2faf7350bdc443faf8beb397b8d" Oct 30 22:19:30.111165 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:30.111112 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27e9c904a3274923a966fa6316e52677df79b2faf7350bdc443faf8beb397b8d"} err="failed to get container status \"27e9c904a3274923a966fa6316e52677df79b2faf7350bdc443faf8beb397b8d\": rpc error: code = NotFound desc = could not find container \"27e9c904a3274923a966fa6316e52677df79b2faf7350bdc443faf8beb397b8d\": container with ID starting with 27e9c904a3274923a966fa6316e52677df79b2faf7350bdc443faf8beb397b8d not found: ID does not exist" Oct 30 22:19:30.146407 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:30.146371 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln"] Oct 30 22:19:30.158651 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:30.158619 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-b9b03-predictor-7f789594bd-xvlln"] Oct 30 22:19:31.131435 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.131402 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:19:31.233995 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.233965 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f52379ef-2e4a-4221-b2ce-fa1e82bdd593" path="/var/lib/kubelet/pods/f52379ef-2e4a-4221-b2ce-fa1e82bdd593/volumes" Oct 30 22:19:31.309538 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.309502 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:19:31.510277 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.510193 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:19:31.541708 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.541678 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z"] Oct 30 22:19:31.542025 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.542013 4615 cpu_manager.go:414] "containerMap: removing container" podUID="f52379ef-2e4a-4221-b2ce-fa1e82bdd593" containerName="kserve-container" Oct 30 22:19:31.542087 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.542026 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f52379ef-2e4a-4221-b2ce-fa1e82bdd593" containerName="kserve-container" Oct 30 22:19:31.542087 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.542038 4615 cpu_manager.go:414] "containerMap: removing container" podUID="f52379ef-2e4a-4221-b2ce-fa1e82bdd593" containerName="kube-rbac-proxy" Oct 30 22:19:31.542087 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.542058 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f52379ef-2e4a-4221-b2ce-fa1e82bdd593" containerName="kube-rbac-proxy" Oct 30 22:19:31.542183 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.542124 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="f52379ef-2e4a-4221-b2ce-fa1e82bdd593" containerName="kserve-container" Oct 30 22:19:31.542183 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.542132 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="f52379ef-2e4a-4221-b2ce-fa1e82bdd593" containerName="kube-rbac-proxy" Oct 30 22:19:31.549156 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.547398 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" Oct 30 22:19:31.555143 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.555121 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b7cd2dea-2cfb-4453-948e-4426ec41109c-proxy-tls\") pod \"error-404-isvc-dbc3c-predictor-549cf4588b-czv9z\" (UID: \"b7cd2dea-2cfb-4453-948e-4426ec41109c\") " pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" Oct 30 22:19:31.555293 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.555160 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7djr\" (UniqueName: \"kubernetes.io/projected/b7cd2dea-2cfb-4453-948e-4426ec41109c-kube-api-access-c7djr\") pod \"error-404-isvc-dbc3c-predictor-549cf4588b-czv9z\" (UID: \"b7cd2dea-2cfb-4453-948e-4426ec41109c\") " pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" Oct 30 22:19:31.555293 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.555193 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b7cd2dea-2cfb-4453-948e-4426ec41109c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-dbc3c-predictor-549cf4588b-czv9z\" (UID: \"b7cd2dea-2cfb-4453-948e-4426ec41109c\") " pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" Oct 30 22:19:31.577148 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.577120 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-cc765036c8\"/\"openshift-service-ca.crt\"" Oct 30 22:19:31.577148 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.577143 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-cc765036c8\"/\"default-dockercfg-r6qmt\"" Oct 30 22:19:31.577352 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.577120 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-cc765036c8\"/\"error-404-isvc-dbc3c-kube-rbac-proxy-sar-config\"" Oct 30 22:19:31.577352 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.577166 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-cc765036c8\"/\"error-404-isvc-dbc3c-predictor-serving-cert\"" Oct 30 22:19:31.577352 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.577207 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-cc765036c8\"/\"kube-root-ca.crt\"" Oct 30 22:19:31.578683 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.578658 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z"] Oct 30 22:19:31.643992 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.643958 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:19:31.656611 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.656591 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b7cd2dea-2cfb-4453-948e-4426ec41109c-proxy-tls\") pod \"error-404-isvc-dbc3c-predictor-549cf4588b-czv9z\" (UID: \"b7cd2dea-2cfb-4453-948e-4426ec41109c\") " pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" Oct 30 22:19:31.656678 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.656622 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7djr\" (UniqueName: \"kubernetes.io/projected/b7cd2dea-2cfb-4453-948e-4426ec41109c-kube-api-access-c7djr\") pod \"error-404-isvc-dbc3c-predictor-549cf4588b-czv9z\" (UID: \"b7cd2dea-2cfb-4453-948e-4426ec41109c\") " pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" Oct 30 22:19:31.656678 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.656644 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b7cd2dea-2cfb-4453-948e-4426ec41109c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-dbc3c-predictor-549cf4588b-czv9z\" (UID: \"b7cd2dea-2cfb-4453-948e-4426ec41109c\") " pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" Oct 30 22:19:31.657276 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.657259 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b7cd2dea-2cfb-4453-948e-4426ec41109c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-dbc3c-predictor-549cf4588b-czv9z\" (UID: \"b7cd2dea-2cfb-4453-948e-4426ec41109c\") " pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" Oct 30 22:19:31.659082 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.659064 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b7cd2dea-2cfb-4453-948e-4426ec41109c-proxy-tls\") pod \"error-404-isvc-dbc3c-predictor-549cf4588b-czv9z\" (UID: \"b7cd2dea-2cfb-4453-948e-4426ec41109c\") " pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" Oct 30 22:19:31.675964 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.675933 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7djr\" (UniqueName: \"kubernetes.io/projected/b7cd2dea-2cfb-4453-948e-4426ec41109c-kube-api-access-c7djr\") pod \"error-404-isvc-dbc3c-predictor-549cf4588b-czv9z\" (UID: \"b7cd2dea-2cfb-4453-948e-4426ec41109c\") " pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" Oct 30 22:19:31.740679 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.740654 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:19:31.864440 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:31.864360 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" Oct 30 22:19:32.154866 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:32.154840 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z"] Oct 30 22:19:32.156265 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:19:32.156232 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7cd2dea_2cfb_4453_948e_4426ec41109c.slice/crio-1911fb8a7a682cc928f534880d6b7f790f75c6e2ca7125fd5c83df669fa451f0 WatchSource:0}: Error finding container 1911fb8a7a682cc928f534880d6b7f790f75c6e2ca7125fd5c83df669fa451f0: Status 404 returned error can't find the container with id 1911fb8a7a682cc928f534880d6b7f790f75c6e2ca7125fd5c83df669fa451f0 Oct 30 22:19:33.114789 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:33.114751 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" event={"ID":"b7cd2dea-2cfb-4453-948e-4426ec41109c","Type":"ContainerStarted","Data":"17f1c1d9d15e1a446c34f7af9c9968a8c8654461b1b06ad1ebc942c84e885181"} Oct 30 22:19:33.115054 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:33.114822 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" event={"ID":"b7cd2dea-2cfb-4453-948e-4426ec41109c","Type":"ContainerStarted","Data":"ee7870d20d5320fa5eebd3677731c75b21b5a31b2d6a1ae0b7578e0076d2408a"} Oct 30 22:19:33.115054 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:33.114852 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" event={"ID":"b7cd2dea-2cfb-4453-948e-4426ec41109c","Type":"ContainerStarted","Data":"1911fb8a7a682cc928f534880d6b7f790f75c6e2ca7125fd5c83df669fa451f0"} Oct 30 22:19:33.115054 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:33.114937 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" Oct 30 22:19:33.154801 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:33.154753 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" podStartSLOduration=2.154738276 podStartE2EDuration="2.154738276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:33.153727775 +0000 UTC m=+690.069414078" watchObservedRunningTime="2026-10-30 22:19:33.154738276 +0000 UTC m=+690.070424580" Oct 30 22:19:34.119288 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:34.119255 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" Oct 30 22:19:34.120586 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:34.120556 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" podUID="b7cd2dea-2cfb-4453-948e-4426ec41109c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused" Oct 30 22:19:35.123608 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:35.123569 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" podUID="b7cd2dea-2cfb-4453-948e-4426ec41109c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused" Oct 30 22:19:40.127996 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:40.127964 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" Oct 30 22:19:40.128421 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:40.128373 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" podUID="b7cd2dea-2cfb-4453-948e-4426ec41109c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused" Oct 30 22:19:49.591058 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:49.591019 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:19:50.129228 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:50.129187 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" podUID="b7cd2dea-2cfb-4453-948e-4426ec41109c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused" Oct 30 22:19:54.375473 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:54.375432 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:19:54.617638 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:19:54.617607 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:19:59.763236 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:19:59.763200 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:20:00.129429 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:00.129341 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" podUID="b7cd2dea-2cfb-4453-948e-4426ec41109c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused" Oct 30 22:20:06.356204 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:06.356167 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h"] Oct 30 22:20:06.356717 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:06.356430 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="kserve-container" containerID="cri-o://e6db15913bd808cd57d4cc0467502eb158570393423cd10b7e68838a01c23197" gracePeriod=30 Oct 30 22:20:06.356717 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:06.356458 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="kube-rbac-proxy" containerID="cri-o://c376c3cdc8c28471d8577f3a3fe73a9262cf14541a94acb36f9b53d85f57038d" gracePeriod=30 Oct 30 22:20:06.417457 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:06.417419 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:20:06.472171 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:06.471241 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx"] Oct 30 22:20:06.472171 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:06.471527 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerName="kserve-container" containerID="cri-o://dcd520aa767fcc521a67aa62d0c8332860060008a51f429008c1d85a9f6039c2" gracePeriod=30 Oct 30 22:20:06.472171 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:06.471866 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerName="kube-rbac-proxy" containerID="cri-o://076b1fbaea312dbad83b2b89a99b6c0ac775fd3a89da8199fc32d92f2a2c4365" gracePeriod=30 Oct 30 22:20:06.473892 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:06.473865 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms"] Oct 30 22:20:06.474172 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:06.474126 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" podUID="c428d17c-304f-44af-9a55-c772082995a2" containerName="kserve-container" containerID="cri-o://78644f4e4325a39481b24c00c1016cc5ced918cdfbfebb3343b830f6d63e597c" gracePeriod=30 Oct 30 22:20:06.474286 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:06.474131 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" podUID="c428d17c-304f-44af-9a55-c772082995a2" containerName="kube-rbac-proxy" containerID="cri-o://be8058ad59e80c000e18d2fb98cddec64089db24822149358a544a6afec0c0fc" gracePeriod=30 Oct 30 22:20:06.552928 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:06.552888 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:20:06.735780 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:06.735735 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused" Oct 30 22:20:06.735984 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:06.735741 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.26:8643/healthz\": dial tcp 10.134.0.26:8643: connect: connection refused" Oct 30 22:20:07.258160 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:07.258120 4615 generic.go:356] "Generic (PLEG): container finished" podID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerID="c376c3cdc8c28471d8577f3a3fe73a9262cf14541a94acb36f9b53d85f57038d" exitCode=0 Oct 30 22:20:07.258336 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:07.258196 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" event={"ID":"bc6485ca-94f8-474c-a7a0-f185f67eebdc","Type":"ContainerDied","Data":"c376c3cdc8c28471d8577f3a3fe73a9262cf14541a94acb36f9b53d85f57038d"} Oct 30 22:20:07.260288 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:07.260266 4615 generic.go:356] "Generic (PLEG): container finished" podID="c428d17c-304f-44af-9a55-c772082995a2" containerID="be8058ad59e80c000e18d2fb98cddec64089db24822149358a544a6afec0c0fc" exitCode=0 Oct 30 22:20:07.260370 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:07.260340 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" event={"ID":"c428d17c-304f-44af-9a55-c772082995a2","Type":"ContainerDied","Data":"be8058ad59e80c000e18d2fb98cddec64089db24822149358a544a6afec0c0fc"} Oct 30 22:20:07.262320 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:07.262301 4615 generic.go:356] "Generic (PLEG): container finished" podID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerID="076b1fbaea312dbad83b2b89a99b6c0ac775fd3a89da8199fc32d92f2a2c4365" exitCode=0 Oct 30 22:20:07.262434 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:07.262326 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" event={"ID":"e27299f9-1e59-4c12-8baa-70f3d276f404","Type":"ContainerDied","Data":"076b1fbaea312dbad83b2b89a99b6c0ac775fd3a89da8199fc32d92f2a2c4365"} Oct 30 22:20:07.762592 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:07.762544 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" podUID="c428d17c-304f-44af-9a55-c772082995a2" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused" Oct 30 22:20:08.229401 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:08.229358 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" podUID="c428d17c-304f-44af-9a55-c772082995a2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Oct 30 22:20:09.228939 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:09.228899 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Oct 30 22:20:10.128426 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:10.128390 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" podUID="b7cd2dea-2cfb-4453-948e-4426ec41109c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused" Oct 30 22:20:10.815639 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:10.815616 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:20:10.989376 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:10.989326 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c428d17c-304f-44af-9a55-c772082995a2-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c428d17c-304f-44af-9a55-c772082995a2-cabundle-cert\") pod \"c428d17c-304f-44af-9a55-c772082995a2\" (UID: \"c428d17c-304f-44af-9a55-c772082995a2\") " Oct 30 22:20:10.989558 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:10.989391 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c428d17c-304f-44af-9a55-c772082995a2-proxy-tls\" (UniqueName: \"kubernetes.io/secret/c428d17c-304f-44af-9a55-c772082995a2-proxy-tls\") pod \"c428d17c-304f-44af-9a55-c772082995a2\" (UID: \"c428d17c-304f-44af-9a55-c772082995a2\") " Oct 30 22:20:10.989558 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:10.989465 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c428d17c-304f-44af-9a55-c772082995a2-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c428d17c-304f-44af-9a55-c772082995a2-kserve-provision-location\") pod \"c428d17c-304f-44af-9a55-c772082995a2\" (UID: \"c428d17c-304f-44af-9a55-c772082995a2\") " Oct 30 22:20:10.989558 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:10.989509 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c428d17c-304f-44af-9a55-c772082995a2-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c428d17c-304f-44af-9a55-c772082995a2-kube-rbac-proxy-sar-config\") pod \"c428d17c-304f-44af-9a55-c772082995a2\" (UID: \"c428d17c-304f-44af-9a55-c772082995a2\") " Oct 30 22:20:10.989732 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:10.989583 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c428d17c-304f-44af-9a55-c772082995a2-kube-api-access-9cmhm\" (UniqueName: \"kubernetes.io/projected/c428d17c-304f-44af-9a55-c772082995a2-kube-api-access-9cmhm\") pod \"c428d17c-304f-44af-9a55-c772082995a2\" (UID: \"c428d17c-304f-44af-9a55-c772082995a2\") " Oct 30 22:20:10.989732 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:10.989713 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c428d17c-304f-44af-9a55-c772082995a2-cabundle-cert" pod "c428d17c-304f-44af-9a55-c772082995a2" (UID: "c428d17c-304f-44af-9a55-c772082995a2"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:20:10.989903 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:10.989758 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c428d17c-304f-44af-9a55-c772082995a2-kserve-provision-location" pod "c428d17c-304f-44af-9a55-c772082995a2" (UID: "c428d17c-304f-44af-9a55-c772082995a2"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:20:10.989953 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:10.989900 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c428d17c-304f-44af-9a55-c772082995a2-kube-rbac-proxy-sar-config" pod "c428d17c-304f-44af-9a55-c772082995a2" (UID: "c428d17c-304f-44af-9a55-c772082995a2"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:20:10.989996 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:10.989973 4615 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c428d17c-304f-44af-9a55-c772082995a2-cabundle-cert\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:20:10.989996 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:10.989985 4615 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c428d17c-304f-44af-9a55-c772082995a2-kserve-provision-location\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:20:10.990111 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:10.989996 4615 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c428d17c-304f-44af-9a55-c772082995a2-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:20:10.991622 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:10.991602 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c428d17c-304f-44af-9a55-c772082995a2-proxy-tls" pod "c428d17c-304f-44af-9a55-c772082995a2" (UID: "c428d17c-304f-44af-9a55-c772082995a2"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:20:10.991730 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:10.991713 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c428d17c-304f-44af-9a55-c772082995a2-kube-api-access-9cmhm" pod "c428d17c-304f-44af-9a55-c772082995a2" (UID: "c428d17c-304f-44af-9a55-c772082995a2"). InnerVolumeSpecName "kube-api-access-9cmhm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:20:11.091095 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.091058 4615 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cmhm\" (UniqueName: \"kubernetes.io/projected/c428d17c-304f-44af-9a55-c772082995a2-kube-api-access-9cmhm\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:20:11.091095 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.091090 4615 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c428d17c-304f-44af-9a55-c772082995a2-proxy-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:20:11.282469 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.282424 4615 generic.go:356] "Generic (PLEG): container finished" podID="c428d17c-304f-44af-9a55-c772082995a2" containerID="78644f4e4325a39481b24c00c1016cc5ced918cdfbfebb3343b830f6d63e597c" exitCode=0 Oct 30 22:20:11.282661 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.282514 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" event={"ID":"c428d17c-304f-44af-9a55-c772082995a2","Type":"ContainerDied","Data":"78644f4e4325a39481b24c00c1016cc5ced918cdfbfebb3343b830f6d63e597c"} Oct 30 22:20:11.282661 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.282601 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" event={"ID":"c428d17c-304f-44af-9a55-c772082995a2","Type":"ContainerDied","Data":"f7f98c62c8e17dd4e26e11703b9e657be62841f76c59b44dc8a70d5d370c3d46"} Oct 30 22:20:11.282661 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.282527 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms" Oct 30 22:20:11.282661 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.282640 4615 scope.go:122] "RemoveContainer" containerID="be8058ad59e80c000e18d2fb98cddec64089db24822149358a544a6afec0c0fc" Oct 30 22:20:11.291106 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.291085 4615 scope.go:122] "RemoveContainer" containerID="78644f4e4325a39481b24c00c1016cc5ced918cdfbfebb3343b830f6d63e597c" Oct 30 22:20:11.298493 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.298474 4615 scope.go:122] "RemoveContainer" containerID="1972b01f1fff6e076c09ba460da8a22df13c26fc58813a4c66571c15cc84659f" Oct 30 22:20:11.305967 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.305943 4615 scope.go:122] "RemoveContainer" containerID="be8058ad59e80c000e18d2fb98cddec64089db24822149358a544a6afec0c0fc" Oct 30 22:20:11.306340 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:20:11.306311 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be8058ad59e80c000e18d2fb98cddec64089db24822149358a544a6afec0c0fc\": container with ID starting with be8058ad59e80c000e18d2fb98cddec64089db24822149358a544a6afec0c0fc not found: ID does not exist" containerID="be8058ad59e80c000e18d2fb98cddec64089db24822149358a544a6afec0c0fc" Oct 30 22:20:11.306418 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.306342 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be8058ad59e80c000e18d2fb98cddec64089db24822149358a544a6afec0c0fc"} err="failed to get container status \"be8058ad59e80c000e18d2fb98cddec64089db24822149358a544a6afec0c0fc\": rpc error: code = NotFound desc = could not find container \"be8058ad59e80c000e18d2fb98cddec64089db24822149358a544a6afec0c0fc\": container with ID starting with be8058ad59e80c000e18d2fb98cddec64089db24822149358a544a6afec0c0fc not found: ID does not exist" Oct 30 22:20:11.306418 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.306360 4615 scope.go:122] "RemoveContainer" containerID="78644f4e4325a39481b24c00c1016cc5ced918cdfbfebb3343b830f6d63e597c" Oct 30 22:20:11.306628 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:20:11.306605 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78644f4e4325a39481b24c00c1016cc5ced918cdfbfebb3343b830f6d63e597c\": container with ID starting with 78644f4e4325a39481b24c00c1016cc5ced918cdfbfebb3343b830f6d63e597c not found: ID does not exist" containerID="78644f4e4325a39481b24c00c1016cc5ced918cdfbfebb3343b830f6d63e597c" Oct 30 22:20:11.306667 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.306635 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78644f4e4325a39481b24c00c1016cc5ced918cdfbfebb3343b830f6d63e597c"} err="failed to get container status \"78644f4e4325a39481b24c00c1016cc5ced918cdfbfebb3343b830f6d63e597c\": rpc error: code = NotFound desc = could not find container \"78644f4e4325a39481b24c00c1016cc5ced918cdfbfebb3343b830f6d63e597c\": container with ID starting with 78644f4e4325a39481b24c00c1016cc5ced918cdfbfebb3343b830f6d63e597c not found: ID does not exist" Oct 30 22:20:11.306667 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.306651 4615 scope.go:122] "RemoveContainer" containerID="1972b01f1fff6e076c09ba460da8a22df13c26fc58813a4c66571c15cc84659f" Oct 30 22:20:11.306853 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:20:11.306836 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1972b01f1fff6e076c09ba460da8a22df13c26fc58813a4c66571c15cc84659f\": container with ID starting with 1972b01f1fff6e076c09ba460da8a22df13c26fc58813a4c66571c15cc84659f not found: ID does not exist" containerID="1972b01f1fff6e076c09ba460da8a22df13c26fc58813a4c66571c15cc84659f" Oct 30 22:20:11.306888 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.306857 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1972b01f1fff6e076c09ba460da8a22df13c26fc58813a4c66571c15cc84659f"} err="failed to get container status \"1972b01f1fff6e076c09ba460da8a22df13c26fc58813a4c66571c15cc84659f\": rpc error: code = NotFound desc = could not find container \"1972b01f1fff6e076c09ba460da8a22df13c26fc58813a4c66571c15cc84659f\": container with ID starting with 1972b01f1fff6e076c09ba460da8a22df13c26fc58813a4c66571c15cc84659f not found: ID does not exist" Oct 30 22:20:11.346997 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.346965 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms"] Oct 30 22:20:11.359173 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.359149 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-1b4518b221/isvc-xgboost-graph-predictor-58fb998f8b-xvnms"] Oct 30 22:20:11.719364 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.719334 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:20:11.797224 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.797198 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:20:11.899492 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.899462 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc6485ca-94f8-474c-a7a0-f185f67eebdc-kube-api-access-kqqtq\" (UniqueName: \"kubernetes.io/projected/bc6485ca-94f8-474c-a7a0-f185f67eebdc-kube-api-access-kqqtq\") pod \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\" (UID: \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\") " Oct 30 22:20:11.899492 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.899500 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e27299f9-1e59-4c12-8baa-70f3d276f404-kube-api-access-q5zwf\" (UniqueName: \"kubernetes.io/projected/e27299f9-1e59-4c12-8baa-70f3d276f404-kube-api-access-q5zwf\") pod \"e27299f9-1e59-4c12-8baa-70f3d276f404\" (UID: \"e27299f9-1e59-4c12-8baa-70f3d276f404\") " Oct 30 22:20:11.900079 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.899538 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc6485ca-94f8-474c-a7a0-f185f67eebdc-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bc6485ca-94f8-474c-a7a0-f185f67eebdc-kube-rbac-proxy-sar-config\") pod \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\" (UID: \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\") " Oct 30 22:20:11.900079 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.899652 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e27299f9-1e59-4c12-8baa-70f3d276f404-proxy-tls\" (UniqueName: \"kubernetes.io/secret/e27299f9-1e59-4c12-8baa-70f3d276f404-proxy-tls\") pod \"e27299f9-1e59-4c12-8baa-70f3d276f404\" (UID: \"e27299f9-1e59-4c12-8baa-70f3d276f404\") " Oct 30 22:20:11.900079 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.899702 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e27299f9-1e59-4c12-8baa-70f3d276f404-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e27299f9-1e59-4c12-8baa-70f3d276f404-kserve-provision-location\") pod \"e27299f9-1e59-4c12-8baa-70f3d276f404\" (UID: \"e27299f9-1e59-4c12-8baa-70f3d276f404\") " Oct 30 22:20:11.900079 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.899751 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc6485ca-94f8-474c-a7a0-f185f67eebdc-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bc6485ca-94f8-474c-a7a0-f185f67eebdc-cabundle-cert\") pod \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\" (UID: \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\") " Oct 30 22:20:11.900079 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.899796 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc6485ca-94f8-474c-a7a0-f185f67eebdc-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bc6485ca-94f8-474c-a7a0-f185f67eebdc-kserve-provision-location\") pod \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\" (UID: \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\") " Oct 30 22:20:11.900079 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.899875 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bc6485ca-94f8-474c-a7a0-f185f67eebdc-proxy-tls\" (UniqueName: \"kubernetes.io/secret/bc6485ca-94f8-474c-a7a0-f185f67eebdc-proxy-tls\") pod \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\" (UID: \"bc6485ca-94f8-474c-a7a0-f185f67eebdc\") " Oct 30 22:20:11.900079 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.899885 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc6485ca-94f8-474c-a7a0-f185f67eebdc-kube-rbac-proxy-sar-config" pod "bc6485ca-94f8-474c-a7a0-f185f67eebdc" (UID: "bc6485ca-94f8-474c-a7a0-f185f67eebdc"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:20:11.900079 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.899905 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e27299f9-1e59-4c12-8baa-70f3d276f404-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e27299f9-1e59-4c12-8baa-70f3d276f404-cabundle-cert\") pod \"e27299f9-1e59-4c12-8baa-70f3d276f404\" (UID: \"e27299f9-1e59-4c12-8baa-70f3d276f404\") " Oct 30 22:20:11.900079 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.899951 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e27299f9-1e59-4c12-8baa-70f3d276f404-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e27299f9-1e59-4c12-8baa-70f3d276f404-kube-rbac-proxy-sar-config\") pod \"e27299f9-1e59-4c12-8baa-70f3d276f404\" (UID: \"e27299f9-1e59-4c12-8baa-70f3d276f404\") " Oct 30 22:20:11.900544 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.900112 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e27299f9-1e59-4c12-8baa-70f3d276f404-kserve-provision-location" pod "e27299f9-1e59-4c12-8baa-70f3d276f404" (UID: "e27299f9-1e59-4c12-8baa-70f3d276f404"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:20:11.900544 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.900137 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc6485ca-94f8-474c-a7a0-f185f67eebdc-cabundle-cert" pod "bc6485ca-94f8-474c-a7a0-f185f67eebdc" (UID: "bc6485ca-94f8-474c-a7a0-f185f67eebdc"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:20:11.900544 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.900265 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc6485ca-94f8-474c-a7a0-f185f67eebdc-kserve-provision-location" pod "bc6485ca-94f8-474c-a7a0-f185f67eebdc" (UID: "bc6485ca-94f8-474c-a7a0-f185f67eebdc"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 22:20:11.900544 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.900423 4615 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bc6485ca-94f8-474c-a7a0-f185f67eebdc-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:20:11.900544 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.900442 4615 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e27299f9-1e59-4c12-8baa-70f3d276f404-kserve-provision-location\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:20:11.900544 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.900456 4615 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bc6485ca-94f8-474c-a7a0-f185f67eebdc-cabundle-cert\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:20:11.900544 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.900459 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e27299f9-1e59-4c12-8baa-70f3d276f404-kube-rbac-proxy-sar-config" pod "e27299f9-1e59-4c12-8baa-70f3d276f404" (UID: "e27299f9-1e59-4c12-8baa-70f3d276f404"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:20:11.900544 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.900472 4615 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bc6485ca-94f8-474c-a7a0-f185f67eebdc-kserve-provision-location\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:20:11.900968 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.900552 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e27299f9-1e59-4c12-8baa-70f3d276f404-cabundle-cert" pod "e27299f9-1e59-4c12-8baa-70f3d276f404" (UID: "e27299f9-1e59-4c12-8baa-70f3d276f404"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:20:11.901893 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.901871 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e27299f9-1e59-4c12-8baa-70f3d276f404-proxy-tls" pod "e27299f9-1e59-4c12-8baa-70f3d276f404" (UID: "e27299f9-1e59-4c12-8baa-70f3d276f404"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:20:11.902000 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.901904 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc6485ca-94f8-474c-a7a0-f185f67eebdc-kube-api-access-kqqtq" pod "bc6485ca-94f8-474c-a7a0-f185f67eebdc" (UID: "bc6485ca-94f8-474c-a7a0-f185f67eebdc"). InnerVolumeSpecName "kube-api-access-kqqtq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:20:11.902000 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.901920 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e27299f9-1e59-4c12-8baa-70f3d276f404-kube-api-access-q5zwf" pod "e27299f9-1e59-4c12-8baa-70f3d276f404" (UID: "e27299f9-1e59-4c12-8baa-70f3d276f404"). InnerVolumeSpecName "kube-api-access-q5zwf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:20:11.902134 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:11.902116 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc6485ca-94f8-474c-a7a0-f185f67eebdc-proxy-tls" pod "bc6485ca-94f8-474c-a7a0-f185f67eebdc" (UID: "bc6485ca-94f8-474c-a7a0-f185f67eebdc"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:20:12.001272 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.001235 4615 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e27299f9-1e59-4c12-8baa-70f3d276f404-proxy-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:20:12.001272 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.001265 4615 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bc6485ca-94f8-474c-a7a0-f185f67eebdc-proxy-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:20:12.001272 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.001274 4615 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e27299f9-1e59-4c12-8baa-70f3d276f404-cabundle-cert\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:20:12.001272 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.001282 4615 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e27299f9-1e59-4c12-8baa-70f3d276f404-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:20:12.001515 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.001293 4615 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqqtq\" (UniqueName: \"kubernetes.io/projected/bc6485ca-94f8-474c-a7a0-f185f67eebdc-kube-api-access-kqqtq\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:20:12.001515 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.001301 4615 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5zwf\" (UniqueName: \"kubernetes.io/projected/e27299f9-1e59-4c12-8baa-70f3d276f404-kube-api-access-q5zwf\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:20:12.288771 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.288683 4615 generic.go:356] "Generic (PLEG): container finished" podID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerID="dcd520aa767fcc521a67aa62d0c8332860060008a51f429008c1d85a9f6039c2" exitCode=0 Oct 30 22:20:12.288771 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.288736 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" Oct 30 22:20:12.289007 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.288768 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" event={"ID":"e27299f9-1e59-4c12-8baa-70f3d276f404","Type":"ContainerDied","Data":"dcd520aa767fcc521a67aa62d0c8332860060008a51f429008c1d85a9f6039c2"} Oct 30 22:20:12.289007 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.288830 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx" event={"ID":"e27299f9-1e59-4c12-8baa-70f3d276f404","Type":"ContainerDied","Data":"47cb0ca8b136a06514b4dfeb737b10dba53882f429a440ae4c30c6a2b490b257"} Oct 30 22:20:12.289007 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.288865 4615 scope.go:122] "RemoveContainer" containerID="076b1fbaea312dbad83b2b89a99b6c0ac775fd3a89da8199fc32d92f2a2c4365" Oct 30 22:20:12.290838 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.290815 4615 generic.go:356] "Generic (PLEG): container finished" podID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerID="e6db15913bd808cd57d4cc0467502eb158570393423cd10b7e68838a01c23197" exitCode=0 Oct 30 22:20:12.290966 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.290863 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" Oct 30 22:20:12.290966 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.290898 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" event={"ID":"bc6485ca-94f8-474c-a7a0-f185f67eebdc","Type":"ContainerDied","Data":"e6db15913bd808cd57d4cc0467502eb158570393423cd10b7e68838a01c23197"} Oct 30 22:20:12.290966 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.290951 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" event={"ID":"bc6485ca-94f8-474c-a7a0-f185f67eebdc","Type":"ContainerDied","Data":"a4d14d3c4123f8a176f7b8c83302255f0d7d11e25bb744f71657c6740426c620"} Oct 30 22:20:12.297465 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.297446 4615 scope.go:122] "RemoveContainer" containerID="dcd520aa767fcc521a67aa62d0c8332860060008a51f429008c1d85a9f6039c2" Oct 30 22:20:12.305923 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.305907 4615 scope.go:122] "RemoveContainer" containerID="d777caa009f7fde17a60bd674364ff4f247fb5f7bfff2eb871f116dd3589c994" Oct 30 22:20:12.312821 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.312803 4615 scope.go:122] "RemoveContainer" containerID="076b1fbaea312dbad83b2b89a99b6c0ac775fd3a89da8199fc32d92f2a2c4365" Oct 30 22:20:12.313077 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:20:12.313037 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"076b1fbaea312dbad83b2b89a99b6c0ac775fd3a89da8199fc32d92f2a2c4365\": container with ID starting with 076b1fbaea312dbad83b2b89a99b6c0ac775fd3a89da8199fc32d92f2a2c4365 not found: ID does not exist" containerID="076b1fbaea312dbad83b2b89a99b6c0ac775fd3a89da8199fc32d92f2a2c4365" Oct 30 22:20:12.313153 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.313091 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"076b1fbaea312dbad83b2b89a99b6c0ac775fd3a89da8199fc32d92f2a2c4365"} err="failed to get container status \"076b1fbaea312dbad83b2b89a99b6c0ac775fd3a89da8199fc32d92f2a2c4365\": rpc error: code = NotFound desc = could not find container \"076b1fbaea312dbad83b2b89a99b6c0ac775fd3a89da8199fc32d92f2a2c4365\": container with ID starting with 076b1fbaea312dbad83b2b89a99b6c0ac775fd3a89da8199fc32d92f2a2c4365 not found: ID does not exist" Oct 30 22:20:12.313153 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.313117 4615 scope.go:122] "RemoveContainer" containerID="dcd520aa767fcc521a67aa62d0c8332860060008a51f429008c1d85a9f6039c2" Oct 30 22:20:12.313384 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:20:12.313366 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcd520aa767fcc521a67aa62d0c8332860060008a51f429008c1d85a9f6039c2\": container with ID starting with dcd520aa767fcc521a67aa62d0c8332860060008a51f429008c1d85a9f6039c2 not found: ID does not exist" containerID="dcd520aa767fcc521a67aa62d0c8332860060008a51f429008c1d85a9f6039c2" Oct 30 22:20:12.313439 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.313392 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcd520aa767fcc521a67aa62d0c8332860060008a51f429008c1d85a9f6039c2"} err="failed to get container status \"dcd520aa767fcc521a67aa62d0c8332860060008a51f429008c1d85a9f6039c2\": rpc error: code = NotFound desc = could not find container \"dcd520aa767fcc521a67aa62d0c8332860060008a51f429008c1d85a9f6039c2\": container with ID starting with dcd520aa767fcc521a67aa62d0c8332860060008a51f429008c1d85a9f6039c2 not found: ID does not exist" Oct 30 22:20:12.313439 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.313412 4615 scope.go:122] "RemoveContainer" containerID="d777caa009f7fde17a60bd674364ff4f247fb5f7bfff2eb871f116dd3589c994" Oct 30 22:20:12.313636 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:20:12.313619 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d777caa009f7fde17a60bd674364ff4f247fb5f7bfff2eb871f116dd3589c994\": container with ID starting with d777caa009f7fde17a60bd674364ff4f247fb5f7bfff2eb871f116dd3589c994 not found: ID does not exist" containerID="d777caa009f7fde17a60bd674364ff4f247fb5f7bfff2eb871f116dd3589c994" Oct 30 22:20:12.313693 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.313644 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d777caa009f7fde17a60bd674364ff4f247fb5f7bfff2eb871f116dd3589c994"} err="failed to get container status \"d777caa009f7fde17a60bd674364ff4f247fb5f7bfff2eb871f116dd3589c994\": rpc error: code = NotFound desc = could not find container \"d777caa009f7fde17a60bd674364ff4f247fb5f7bfff2eb871f116dd3589c994\": container with ID starting with d777caa009f7fde17a60bd674364ff4f247fb5f7bfff2eb871f116dd3589c994 not found: ID does not exist" Oct 30 22:20:12.313693 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.313664 4615 scope.go:122] "RemoveContainer" containerID="c376c3cdc8c28471d8577f3a3fe73a9262cf14541a94acb36f9b53d85f57038d" Oct 30 22:20:12.320744 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.320728 4615 scope.go:122] "RemoveContainer" containerID="e6db15913bd808cd57d4cc0467502eb158570393423cd10b7e68838a01c23197" Oct 30 22:20:12.327691 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.327671 4615 scope.go:122] "RemoveContainer" containerID="688452f551fbd3a080057645b984502837c957f9be1d86b4d37c406131c3b18c" Oct 30 22:20:12.335010 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.334993 4615 scope.go:122] "RemoveContainer" containerID="c376c3cdc8c28471d8577f3a3fe73a9262cf14541a94acb36f9b53d85f57038d" Oct 30 22:20:12.335275 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:20:12.335253 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c376c3cdc8c28471d8577f3a3fe73a9262cf14541a94acb36f9b53d85f57038d\": container with ID starting with c376c3cdc8c28471d8577f3a3fe73a9262cf14541a94acb36f9b53d85f57038d not found: ID does not exist" containerID="c376c3cdc8c28471d8577f3a3fe73a9262cf14541a94acb36f9b53d85f57038d" Oct 30 22:20:12.335353 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.335288 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c376c3cdc8c28471d8577f3a3fe73a9262cf14541a94acb36f9b53d85f57038d"} err="failed to get container status \"c376c3cdc8c28471d8577f3a3fe73a9262cf14541a94acb36f9b53d85f57038d\": rpc error: code = NotFound desc = could not find container \"c376c3cdc8c28471d8577f3a3fe73a9262cf14541a94acb36f9b53d85f57038d\": container with ID starting with c376c3cdc8c28471d8577f3a3fe73a9262cf14541a94acb36f9b53d85f57038d not found: ID does not exist" Oct 30 22:20:12.335353 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.335311 4615 scope.go:122] "RemoveContainer" containerID="e6db15913bd808cd57d4cc0467502eb158570393423cd10b7e68838a01c23197" Oct 30 22:20:12.335552 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:20:12.335533 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6db15913bd808cd57d4cc0467502eb158570393423cd10b7e68838a01c23197\": container with ID starting with e6db15913bd808cd57d4cc0467502eb158570393423cd10b7e68838a01c23197 not found: ID does not exist" containerID="e6db15913bd808cd57d4cc0467502eb158570393423cd10b7e68838a01c23197" Oct 30 22:20:12.335587 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.335560 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6db15913bd808cd57d4cc0467502eb158570393423cd10b7e68838a01c23197"} err="failed to get container status \"e6db15913bd808cd57d4cc0467502eb158570393423cd10b7e68838a01c23197\": rpc error: code = NotFound desc = could not find container \"e6db15913bd808cd57d4cc0467502eb158570393423cd10b7e68838a01c23197\": container with ID starting with e6db15913bd808cd57d4cc0467502eb158570393423cd10b7e68838a01c23197 not found: ID does not exist" Oct 30 22:20:12.335587 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.335577 4615 scope.go:122] "RemoveContainer" containerID="688452f551fbd3a080057645b984502837c957f9be1d86b4d37c406131c3b18c" Oct 30 22:20:12.335835 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:20:12.335815 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"688452f551fbd3a080057645b984502837c957f9be1d86b4d37c406131c3b18c\": container with ID starting with 688452f551fbd3a080057645b984502837c957f9be1d86b4d37c406131c3b18c not found: ID does not exist" containerID="688452f551fbd3a080057645b984502837c957f9be1d86b4d37c406131c3b18c" Oct 30 22:20:12.335881 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.335841 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"688452f551fbd3a080057645b984502837c957f9be1d86b4d37c406131c3b18c"} err="failed to get container status \"688452f551fbd3a080057645b984502837c957f9be1d86b4d37c406131c3b18c\": rpc error: code = NotFound desc = could not find container \"688452f551fbd3a080057645b984502837c957f9be1d86b4d37c406131c3b18c\": container with ID starting with 688452f551fbd3a080057645b984502837c957f9be1d86b4d37c406131c3b18c not found: ID does not exist" Oct 30 22:20:12.344374 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.344349 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx"] Oct 30 22:20:12.356619 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.356593 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-1b4518b221/isvc-sklearn-graph-2-predictor-79577dc58b-djdnx"] Oct 30 22:20:12.382123 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.382091 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h"] Oct 30 22:20:12.390653 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.390626 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h"] Oct 30 22:20:12.735214 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.735168 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.26:8643/healthz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 22:20:12.741380 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:12.741344 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/isvc-sklearn-graph-1-predictor-7d6f88d7bf-xzx6h" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: i/o timeout" Oct 30 22:20:13.233182 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:13.233151 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" path="/var/lib/kubelet/pods/bc6485ca-94f8-474c-a7a0-f185f67eebdc/volumes" Oct 30 22:20:13.233632 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:13.233619 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c428d17c-304f-44af-9a55-c772082995a2" path="/var/lib/kubelet/pods/c428d17c-304f-44af-9a55-c772082995a2/volumes" Oct 30 22:20:13.234163 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:13.234143 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" path="/var/lib/kubelet/pods/e27299f9-1e59-4c12-8baa-70f3d276f404/volumes" Oct 30 22:20:13.742714 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:13.742664 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:20:13.912023 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:13.911969 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:20:14.145645 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.145611 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:20:14.191056 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191021 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64"] Oct 30 22:20:14.191381 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191368 4615 cpu_manager.go:414] "containerMap: removing container" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerName="kserve-container" Oct 30 22:20:14.191420 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191383 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerName="kserve-container" Oct 30 22:20:14.191420 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191394 4615 cpu_manager.go:414] "containerMap: removing container" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerName="kube-rbac-proxy" Oct 30 22:20:14.191420 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191402 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerName="kube-rbac-proxy" Oct 30 22:20:14.191420 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191413 4615 cpu_manager.go:414] "containerMap: removing container" podUID="c428d17c-304f-44af-9a55-c772082995a2" containerName="kserve-container" Oct 30 22:20:14.191420 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191419 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c428d17c-304f-44af-9a55-c772082995a2" containerName="kserve-container" Oct 30 22:20:14.191570 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191430 4615 cpu_manager.go:414] "containerMap: removing container" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerName="storage-initializer" Oct 30 22:20:14.191570 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191436 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerName="storage-initializer" Oct 30 22:20:14.191570 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191442 4615 cpu_manager.go:414] "containerMap: removing container" podUID="c428d17c-304f-44af-9a55-c772082995a2" containerName="kube-rbac-proxy" Oct 30 22:20:14.191570 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191447 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c428d17c-304f-44af-9a55-c772082995a2" containerName="kube-rbac-proxy" Oct 30 22:20:14.191570 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191460 4615 cpu_manager.go:414] "containerMap: removing container" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="kube-rbac-proxy" Oct 30 22:20:14.191570 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191464 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="kube-rbac-proxy" Oct 30 22:20:14.191570 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191470 4615 cpu_manager.go:414] "containerMap: removing container" podUID="c428d17c-304f-44af-9a55-c772082995a2" containerName="storage-initializer" Oct 30 22:20:14.191570 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191476 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c428d17c-304f-44af-9a55-c772082995a2" containerName="storage-initializer" Oct 30 22:20:14.191570 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191485 4615 cpu_manager.go:414] "containerMap: removing container" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="kserve-container" Oct 30 22:20:14.191570 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191492 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="kserve-container" Oct 30 22:20:14.191570 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191500 4615 cpu_manager.go:414] "containerMap: removing container" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="storage-initializer" Oct 30 22:20:14.191570 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191504 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="storage-initializer" Oct 30 22:20:14.191570 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191558 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerName="kserve-container" Oct 30 22:20:14.191570 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191570 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="e27299f9-1e59-4c12-8baa-70f3d276f404" containerName="kube-rbac-proxy" Oct 30 22:20:14.191972 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191577 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="c428d17c-304f-44af-9a55-c772082995a2" containerName="kserve-container" Oct 30 22:20:14.191972 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191582 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="c428d17c-304f-44af-9a55-c772082995a2" containerName="kube-rbac-proxy" Oct 30 22:20:14.191972 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191589 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="kube-rbac-proxy" Oct 30 22:20:14.191972 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.191595 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc6485ca-94f8-474c-a7a0-f185f67eebdc" containerName="kserve-container" Oct 30 22:20:14.196641 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.196620 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" Oct 30 22:20:14.211565 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.211545 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-1b4518b221\"/\"default-dockercfg-zht6h\"" Oct 30 22:20:14.223826 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.223797 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1b4518b221\"/\"openshift-service-ca.crt\"" Oct 30 22:20:14.223826 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.223783 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1b4518b221\"/\"error-404-isvc-94a5a-kube-rbac-proxy-sar-config\"" Oct 30 22:20:14.224032 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.223871 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1b4518b221\"/\"kube-root-ca.crt\"" Oct 30 22:20:14.224137 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.223815 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-1b4518b221\"/\"error-404-isvc-94a5a-predictor-serving-cert\"" Oct 30 22:20:14.225090 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.225072 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64"] Oct 30 22:20:14.263331 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.263300 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:20:14.319856 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.319821 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aa9e10f4-f664-4910-bffe-7a9783520fe8-proxy-tls\") pod \"error-404-isvc-94a5a-predictor-9c964b699-9jp64\" (UID: \"aa9e10f4-f664-4910-bffe-7a9783520fe8\") " pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" Oct 30 22:20:14.320035 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.319866 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aa9e10f4-f664-4910-bffe-7a9783520fe8-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-94a5a-predictor-9c964b699-9jp64\" (UID: \"aa9e10f4-f664-4910-bffe-7a9783520fe8\") " pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" Oct 30 22:20:14.320035 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.319972 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thgbb\" (UniqueName: \"kubernetes.io/projected/aa9e10f4-f664-4910-bffe-7a9783520fe8-kube-api-access-thgbb\") pod \"error-404-isvc-94a5a-predictor-9c964b699-9jp64\" (UID: \"aa9e10f4-f664-4910-bffe-7a9783520fe8\") " pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" Oct 30 22:20:14.421261 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.421168 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thgbb\" (UniqueName: \"kubernetes.io/projected/aa9e10f4-f664-4910-bffe-7a9783520fe8-kube-api-access-thgbb\") pod \"error-404-isvc-94a5a-predictor-9c964b699-9jp64\" (UID: \"aa9e10f4-f664-4910-bffe-7a9783520fe8\") " pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" Oct 30 22:20:14.421447 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.421266 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aa9e10f4-f664-4910-bffe-7a9783520fe8-proxy-tls\") pod \"error-404-isvc-94a5a-predictor-9c964b699-9jp64\" (UID: \"aa9e10f4-f664-4910-bffe-7a9783520fe8\") " pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" Oct 30 22:20:14.421447 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.421299 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aa9e10f4-f664-4910-bffe-7a9783520fe8-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-94a5a-predictor-9c964b699-9jp64\" (UID: \"aa9e10f4-f664-4910-bffe-7a9783520fe8\") " pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" Oct 30 22:20:14.421903 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.421881 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aa9e10f4-f664-4910-bffe-7a9783520fe8-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-94a5a-predictor-9c964b699-9jp64\" (UID: \"aa9e10f4-f664-4910-bffe-7a9783520fe8\") " pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" Oct 30 22:20:14.423696 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.423678 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aa9e10f4-f664-4910-bffe-7a9783520fe8-proxy-tls\") pod \"error-404-isvc-94a5a-predictor-9c964b699-9jp64\" (UID: \"aa9e10f4-f664-4910-bffe-7a9783520fe8\") " pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" Oct 30 22:20:14.441020 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.440989 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thgbb\" (UniqueName: \"kubernetes.io/projected/aa9e10f4-f664-4910-bffe-7a9783520fe8-kube-api-access-thgbb\") pod \"error-404-isvc-94a5a-predictor-9c964b699-9jp64\" (UID: \"aa9e10f4-f664-4910-bffe-7a9783520fe8\") " pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" Oct 30 22:20:14.511617 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.511571 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" Oct 30 22:20:14.803528 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:14.803502 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64"] Oct 30 22:20:14.805113 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:20:14.805087 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa9e10f4_f664_4910_bffe_7a9783520fe8.slice/crio-f49dc17b44810fc338299ac727eef4ea93cf2f6cd143ccb070e0a480a5b7cd34 WatchSource:0}: Error finding container f49dc17b44810fc338299ac727eef4ea93cf2f6cd143ccb070e0a480a5b7cd34: Status 404 returned error can't find the container with id f49dc17b44810fc338299ac727eef4ea93cf2f6cd143ccb070e0a480a5b7cd34 Oct 30 22:20:15.308261 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:15.308225 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" event={"ID":"aa9e10f4-f664-4910-bffe-7a9783520fe8","Type":"ContainerStarted","Data":"75c0022995fb2b28bb75a782b8fb15d2e5a1a5efd9871b85e7a542bfbbb43cbb"} Oct 30 22:20:15.308709 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:15.308280 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" event={"ID":"aa9e10f4-f664-4910-bffe-7a9783520fe8","Type":"ContainerStarted","Data":"2ee6b98c5058b3bb393c6501097ad8189ece03716611dec2cb9514cc7ce31859"} Oct 30 22:20:15.308709 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:15.308302 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" event={"ID":"aa9e10f4-f664-4910-bffe-7a9783520fe8","Type":"ContainerStarted","Data":"f49dc17b44810fc338299ac727eef4ea93cf2f6cd143ccb070e0a480a5b7cd34"} Oct 30 22:20:15.308709 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:15.308324 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" Oct 30 22:20:15.308709 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:15.308344 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" Oct 30 22:20:15.309623 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:15.309596 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" podUID="aa9e10f4-f664-4910-bffe-7a9783520fe8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused" Oct 30 22:20:15.348277 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:15.348219 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" podStartSLOduration=1.348204148 podStartE2EDuration="1.348204148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:20:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:20:15.346738344 +0000 UTC m=+732.262424649" watchObservedRunningTime="2026-10-30 22:20:15.348204148 +0000 UTC m=+732.263890442" Oct 30 22:20:16.312825 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:16.312784 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" podUID="aa9e10f4-f664-4910-bffe-7a9783520fe8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused" Oct 30 22:20:20.129453 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:20.129404 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" podUID="b7cd2dea-2cfb-4453-948e-4426ec41109c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused" Oct 30 22:20:21.317146 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:21.317113 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" Oct 30 22:20:21.317622 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:21.317594 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" podUID="aa9e10f4-f664-4910-bffe-7a9783520fe8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused" Oct 30 22:20:29.763205 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:20:29.763145 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:20:30.129504 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:30.129426 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" Oct 30 22:20:31.318136 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:31.318095 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" podUID="aa9e10f4-f664-4910-bffe-7a9783520fe8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused" Oct 30 22:20:41.274793 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:41.274739 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:20:41.317859 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:41.317812 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" podUID="aa9e10f4-f664-4910-bffe-7a9783520fe8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused" Oct 30 22:20:41.378297 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:41.378261 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:20:49.677308 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:49.677218 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:20:51.318245 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:20:51.318198 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" podUID="aa9e10f4-f664-4910-bffe-7a9783520fe8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused" Oct 30 22:20:59.763768 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:20:59.763722 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:21:01.318128 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:01.318089 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" podUID="aa9e10f4-f664-4910-bffe-7a9783520fe8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused" Oct 30 22:21:11.318215 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:11.318187 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" Oct 30 22:21:23.807212 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:23.807176 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t"] Oct 30 22:21:23.810596 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:23.810571 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" Oct 30 22:21:23.831211 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:23.831182 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1b4518b221\"/\"sequence-graph-94a5a-kube-rbac-proxy-sar-config\"" Oct 30 22:21:23.834569 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:23.834545 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:21:23.843332 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:23.843299 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-1b4518b221\"/\"sequence-graph-94a5a-serving-cert\"" Oct 30 22:21:23.844627 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:23.844605 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t"] Oct 30 22:21:23.883007 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:23.882969 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241c8943-6710-48e5-b281-454b94b970be-openshift-service-ca-bundle\") pod \"sequence-graph-94a5a-5bcd4597f4-krr2t\" (UID: \"241c8943-6710-48e5-b281-454b94b970be\") " pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" Oct 30 22:21:23.883007 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:23.883012 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/241c8943-6710-48e5-b281-454b94b970be-proxy-tls\") pod \"sequence-graph-94a5a-5bcd4597f4-krr2t\" (UID: \"241c8943-6710-48e5-b281-454b94b970be\") " pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" Oct 30 22:21:23.983974 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:23.983932 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241c8943-6710-48e5-b281-454b94b970be-openshift-service-ca-bundle\") pod \"sequence-graph-94a5a-5bcd4597f4-krr2t\" (UID: \"241c8943-6710-48e5-b281-454b94b970be\") " pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" Oct 30 22:21:23.984173 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:23.983990 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/241c8943-6710-48e5-b281-454b94b970be-proxy-tls\") pod \"sequence-graph-94a5a-5bcd4597f4-krr2t\" (UID: \"241c8943-6710-48e5-b281-454b94b970be\") " pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" Oct 30 22:21:23.984583 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:23.984550 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241c8943-6710-48e5-b281-454b94b970be-openshift-service-ca-bundle\") pod \"sequence-graph-94a5a-5bcd4597f4-krr2t\" (UID: \"241c8943-6710-48e5-b281-454b94b970be\") " pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" Oct 30 22:21:23.986306 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:23.986288 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/241c8943-6710-48e5-b281-454b94b970be-proxy-tls\") pod \"sequence-graph-94a5a-5bcd4597f4-krr2t\" (UID: \"241c8943-6710-48e5-b281-454b94b970be\") " pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" Oct 30 22:21:24.057882 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:24.057802 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:21:24.123583 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:24.123553 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" Oct 30 22:21:24.413069 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:24.413021 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t"] Oct 30 22:21:24.414318 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:21:24.414294 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod241c8943_6710_48e5_b281_454b94b970be.slice/crio-72b821d29eacb934408c91dfe6d2b9f3d7ee1a2b5205e204039fa967702c80fc WatchSource:0}: Error finding container 72b821d29eacb934408c91dfe6d2b9f3d7ee1a2b5205e204039fa967702c80fc: Status 404 returned error can't find the container with id 72b821d29eacb934408c91dfe6d2b9f3d7ee1a2b5205e204039fa967702c80fc Oct 30 22:21:24.416572 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:24.416554 4615 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 22:21:24.586958 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:24.586923 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" event={"ID":"241c8943-6710-48e5-b281-454b94b970be","Type":"ContainerStarted","Data":"72b821d29eacb934408c91dfe6d2b9f3d7ee1a2b5205e204039fa967702c80fc"} Oct 30 22:21:24.734120 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:24.734010 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:21:26.596855 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:26.596774 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" event={"ID":"241c8943-6710-48e5-b281-454b94b970be","Type":"ContainerStarted","Data":"cf2ebea75ca962372265657be43ee22349e25b04e4992fd628487c637433a4f8"} Oct 30 22:21:26.597247 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:26.596914 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" Oct 30 22:21:26.635877 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:26.635835 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" podStartSLOduration=1.8011574430000001 podStartE2EDuration="3.635822505s" totalImagesPullingTime="1.834665062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:21:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:21:26.63521697 +0000 UTC m=+803.550903273" watchObservedRunningTime="2026-10-30 22:21:26.635822505 +0000 UTC m=+803.551508806" Oct 30 22:21:29.762812 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:21:29.762776 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:21:32.606294 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:32.606264 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" Oct 30 22:21:49.796397 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:49.796358 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:21:55.824944 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:55.824908 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:21:55.957323 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:21:55.957290 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:21:59.762526 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:21:59.762491 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:22:29.762598 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:22:29.762558 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:22:38.935582 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:22:38.935549 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:22:49.860119 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:22:49.860085 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:22:59.763406 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:22:59.763366 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:23:03.323737 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:23:03.323708 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:23:03.346787 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:23:03.346762 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:23:29.762865 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:23:29.762824 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:23:50.022102 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:23:50.022005 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:23:59.768431 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:23:59.768386 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:24:29.764769 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:24:29.764730 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:24:50.136380 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:24:50.136344 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:24:59.762998 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:24:59.762954 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:25:09.751725 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:25:09.751693 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:25:29.763266 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:25:29.763188 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:25:31.180487 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:25:31.180443 4615 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" podUID="f39ce36b79a5ed24b44c91f0e6e003f7" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 22:25:50.222099 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:25:50.222038 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:25:59.761677 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:25:59.761636 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:26:29.764000 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:26:29.763957 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:26:50.331630 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:26:50.331551 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:26:59.763171 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:26:59.763133 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:27:29.763491 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:27:29.763450 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:27:31.179797 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:27:31.179746 4615 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" podUID="f39ce36b79a5ed24b44c91f0e6e003f7" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": context deadline exceeded" Oct 30 22:27:50.502583 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:27:50.502543 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:27:57.239657 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:27:57.239626 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:27:59.762863 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:27:59.762825 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:28:03.350379 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:28:03.350353 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:28:03.370734 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:28:03.370707 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:28:29.763148 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:28:29.763026 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:28:50.570279 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:28:50.570232 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:28:56.694473 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:28:56.694435 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z"] Oct 30 22:28:56.694892 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:28:56.694726 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" podUID="b7cd2dea-2cfb-4453-948e-4426ec41109c" containerName="kserve-container" containerID="cri-o://ee7870d20d5320fa5eebd3677731c75b21b5a31b2d6a1ae0b7578e0076d2408a" gracePeriod=30 Oct 30 22:28:56.694892 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:28:56.694757 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" podUID="b7cd2dea-2cfb-4453-948e-4426ec41109c" containerName="kube-rbac-proxy" containerID="cri-o://17f1c1d9d15e1a446c34f7af9c9968a8c8654461b1b06ad1ebc942c84e885181" gracePeriod=30 Oct 30 22:28:56.758888 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:28:56.758853 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:28:56.871575 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:28:56.871543 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:28:57.417117 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:28:57.417082 4615 generic.go:356] "Generic (PLEG): container finished" podID="b7cd2dea-2cfb-4453-948e-4426ec41109c" containerID="17f1c1d9d15e1a446c34f7af9c9968a8c8654461b1b06ad1ebc942c84e885181" exitCode=0 Oct 30 22:28:57.417292 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:28:57.417163 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" event={"ID":"b7cd2dea-2cfb-4453-948e-4426ec41109c","Type":"ContainerDied","Data":"17f1c1d9d15e1a446c34f7af9c9968a8c8654461b1b06ad1ebc942c84e885181"} Oct 30 22:28:59.762663 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:28:59.762620 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:28:59.945988 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:28:59.945963 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" Oct 30 22:28:59.995237 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:28:59.995205 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b7cd2dea-2cfb-4453-948e-4426ec41109c-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b7cd2dea-2cfb-4453-948e-4426ec41109c-kube-rbac-proxy-sar-config\") pod \"b7cd2dea-2cfb-4453-948e-4426ec41109c\" (UID: \"b7cd2dea-2cfb-4453-948e-4426ec41109c\") " Oct 30 22:28:59.995426 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:28:59.995261 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b7cd2dea-2cfb-4453-948e-4426ec41109c-kube-api-access-c7djr\" (UniqueName: \"kubernetes.io/projected/b7cd2dea-2cfb-4453-948e-4426ec41109c-kube-api-access-c7djr\") pod \"b7cd2dea-2cfb-4453-948e-4426ec41109c\" (UID: \"b7cd2dea-2cfb-4453-948e-4426ec41109c\") " Oct 30 22:28:59.995426 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:28:59.995328 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b7cd2dea-2cfb-4453-948e-4426ec41109c-proxy-tls\" (UniqueName: \"kubernetes.io/secret/b7cd2dea-2cfb-4453-948e-4426ec41109c-proxy-tls\") pod \"b7cd2dea-2cfb-4453-948e-4426ec41109c\" (UID: \"b7cd2dea-2cfb-4453-948e-4426ec41109c\") " Oct 30 22:28:59.995609 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:28:59.995584 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7cd2dea-2cfb-4453-948e-4426ec41109c-kube-rbac-proxy-sar-config" pod "b7cd2dea-2cfb-4453-948e-4426ec41109c" (UID: "b7cd2dea-2cfb-4453-948e-4426ec41109c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:28:59.997385 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:28:59.997357 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7cd2dea-2cfb-4453-948e-4426ec41109c-proxy-tls" pod "b7cd2dea-2cfb-4453-948e-4426ec41109c" (UID: "b7cd2dea-2cfb-4453-948e-4426ec41109c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:28:59.997498 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:28:59.997440 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7cd2dea-2cfb-4453-948e-4426ec41109c-kube-api-access-c7djr" pod "b7cd2dea-2cfb-4453-948e-4426ec41109c" (UID: "b7cd2dea-2cfb-4453-948e-4426ec41109c"). InnerVolumeSpecName "kube-api-access-c7djr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:29:00.096223 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:00.096139 4615 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b7cd2dea-2cfb-4453-948e-4426ec41109c-proxy-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:29:00.096223 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:00.096170 4615 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b7cd2dea-2cfb-4453-948e-4426ec41109c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:29:00.096223 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:00.096179 4615 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7djr\" (UniqueName: \"kubernetes.io/projected/b7cd2dea-2cfb-4453-948e-4426ec41109c-kube-api-access-c7djr\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:29:00.430814 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:00.430773 4615 generic.go:356] "Generic (PLEG): container finished" podID="b7cd2dea-2cfb-4453-948e-4426ec41109c" containerID="ee7870d20d5320fa5eebd3677731c75b21b5a31b2d6a1ae0b7578e0076d2408a" exitCode=0 Oct 30 22:29:00.430986 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:00.430832 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" Oct 30 22:29:00.430986 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:00.430839 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" event={"ID":"b7cd2dea-2cfb-4453-948e-4426ec41109c","Type":"ContainerDied","Data":"ee7870d20d5320fa5eebd3677731c75b21b5a31b2d6a1ae0b7578e0076d2408a"} Oct 30 22:29:00.430986 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:00.430901 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z" event={"ID":"b7cd2dea-2cfb-4453-948e-4426ec41109c","Type":"ContainerDied","Data":"1911fb8a7a682cc928f534880d6b7f790f75c6e2ca7125fd5c83df669fa451f0"} Oct 30 22:29:00.430986 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:00.430939 4615 scope.go:122] "RemoveContainer" containerID="17f1c1d9d15e1a446c34f7af9c9968a8c8654461b1b06ad1ebc942c84e885181" Oct 30 22:29:00.439186 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:00.439163 4615 scope.go:122] "RemoveContainer" containerID="ee7870d20d5320fa5eebd3677731c75b21b5a31b2d6a1ae0b7578e0076d2408a" Oct 30 22:29:00.446300 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:00.446284 4615 scope.go:122] "RemoveContainer" containerID="17f1c1d9d15e1a446c34f7af9c9968a8c8654461b1b06ad1ebc942c84e885181" Oct 30 22:29:00.446597 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:29:00.446577 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17f1c1d9d15e1a446c34f7af9c9968a8c8654461b1b06ad1ebc942c84e885181\": container with ID starting with 17f1c1d9d15e1a446c34f7af9c9968a8c8654461b1b06ad1ebc942c84e885181 not found: ID does not exist" containerID="17f1c1d9d15e1a446c34f7af9c9968a8c8654461b1b06ad1ebc942c84e885181" Oct 30 22:29:00.446646 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:00.446608 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17f1c1d9d15e1a446c34f7af9c9968a8c8654461b1b06ad1ebc942c84e885181"} err="failed to get container status \"17f1c1d9d15e1a446c34f7af9c9968a8c8654461b1b06ad1ebc942c84e885181\": rpc error: code = NotFound desc = could not find container \"17f1c1d9d15e1a446c34f7af9c9968a8c8654461b1b06ad1ebc942c84e885181\": container with ID starting with 17f1c1d9d15e1a446c34f7af9c9968a8c8654461b1b06ad1ebc942c84e885181 not found: ID does not exist" Oct 30 22:29:00.446646 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:00.446625 4615 scope.go:122] "RemoveContainer" containerID="ee7870d20d5320fa5eebd3677731c75b21b5a31b2d6a1ae0b7578e0076d2408a" Oct 30 22:29:00.446857 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:29:00.446839 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee7870d20d5320fa5eebd3677731c75b21b5a31b2d6a1ae0b7578e0076d2408a\": container with ID starting with ee7870d20d5320fa5eebd3677731c75b21b5a31b2d6a1ae0b7578e0076d2408a not found: ID does not exist" containerID="ee7870d20d5320fa5eebd3677731c75b21b5a31b2d6a1ae0b7578e0076d2408a" Oct 30 22:29:00.446917 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:00.446868 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee7870d20d5320fa5eebd3677731c75b21b5a31b2d6a1ae0b7578e0076d2408a"} err="failed to get container status \"ee7870d20d5320fa5eebd3677731c75b21b5a31b2d6a1ae0b7578e0076d2408a\": rpc error: code = NotFound desc = could not find container \"ee7870d20d5320fa5eebd3677731c75b21b5a31b2d6a1ae0b7578e0076d2408a\": container with ID starting with ee7870d20d5320fa5eebd3677731c75b21b5a31b2d6a1ae0b7578e0076d2408a not found: ID does not exist" Oct 30 22:29:00.483190 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:00.483158 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z"] Oct 30 22:29:00.495651 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:00.495627 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-dbc3c-predictor-549cf4588b-czv9z"] Oct 30 22:29:01.232858 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:01.232824 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b7cd2dea-2cfb-4453-948e-4426ec41109c" path="/var/lib/kubelet/pods/b7cd2dea-2cfb-4453-948e-4426ec41109c/volumes" Oct 30 22:29:02.129552 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.129517 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:29:02.303899 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.303860 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:29:02.553788 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.553751 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:29:02.595966 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.595929 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm"] Oct 30 22:29:02.596311 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.596297 4615 cpu_manager.go:414] "containerMap: removing container" podUID="b7cd2dea-2cfb-4453-948e-4426ec41109c" containerName="kube-rbac-proxy" Oct 30 22:29:02.596359 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.596312 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b7cd2dea-2cfb-4453-948e-4426ec41109c" containerName="kube-rbac-proxy" Oct 30 22:29:02.596359 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.596327 4615 cpu_manager.go:414] "containerMap: removing container" podUID="b7cd2dea-2cfb-4453-948e-4426ec41109c" containerName="kserve-container" Oct 30 22:29:02.596359 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.596332 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b7cd2dea-2cfb-4453-948e-4426ec41109c" containerName="kserve-container" Oct 30 22:29:02.596456 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.596393 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="b7cd2dea-2cfb-4453-948e-4426ec41109c" containerName="kube-rbac-proxy" Oct 30 22:29:02.596456 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.596401 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="b7cd2dea-2cfb-4453-948e-4426ec41109c" containerName="kserve-container" Oct 30 22:29:02.600481 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.600453 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" Oct 30 22:29:02.614740 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.614716 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv6vz\" (UniqueName: \"kubernetes.io/projected/d33a90f4-47d2-4445-b644-decbd1626039-kube-api-access-xv6vz\") pod \"error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm\" (UID: \"d33a90f4-47d2-4445-b644-decbd1626039\") " pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" Oct 30 22:29:02.614843 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.614750 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d33a90f4-47d2-4445-b644-decbd1626039-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm\" (UID: \"d33a90f4-47d2-4445-b644-decbd1626039\") " pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" Oct 30 22:29:02.614843 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.614819 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d33a90f4-47d2-4445-b644-decbd1626039-proxy-tls\") pod \"error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm\" (UID: \"d33a90f4-47d2-4445-b644-decbd1626039\") " pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" Oct 30 22:29:02.629566 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.629536 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-cc765036c8\"/\"default-dockercfg-r6qmt\"" Oct 30 22:29:02.629734 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.629627 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-cc765036c8\"/\"error-404-isvc-c1ed1-kube-rbac-proxy-sar-config\"" Oct 30 22:29:02.629734 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.629643 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-cc765036c8\"/\"kube-root-ca.crt\"" Oct 30 22:29:02.629734 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.629659 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-cc765036c8\"/\"openshift-service-ca.crt\"" Oct 30 22:29:02.629868 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.629757 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-cc765036c8\"/\"error-404-isvc-c1ed1-predictor-serving-cert\"" Oct 30 22:29:02.630796 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.630774 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm"] Oct 30 22:29:02.666436 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.666411 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:29:02.716275 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.716242 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d33a90f4-47d2-4445-b644-decbd1626039-proxy-tls\") pod \"error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm\" (UID: \"d33a90f4-47d2-4445-b644-decbd1626039\") " pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" Oct 30 22:29:02.716438 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.716304 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv6vz\" (UniqueName: \"kubernetes.io/projected/d33a90f4-47d2-4445-b644-decbd1626039-kube-api-access-xv6vz\") pod \"error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm\" (UID: \"d33a90f4-47d2-4445-b644-decbd1626039\") " pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" Oct 30 22:29:02.716438 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.716333 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d33a90f4-47d2-4445-b644-decbd1626039-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm\" (UID: \"d33a90f4-47d2-4445-b644-decbd1626039\") " pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" Oct 30 22:29:02.717003 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.716984 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d33a90f4-47d2-4445-b644-decbd1626039-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm\" (UID: \"d33a90f4-47d2-4445-b644-decbd1626039\") " pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" Oct 30 22:29:02.718596 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.718575 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d33a90f4-47d2-4445-b644-decbd1626039-proxy-tls\") pod \"error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm\" (UID: \"d33a90f4-47d2-4445-b644-decbd1626039\") " pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" Oct 30 22:29:02.735907 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.735882 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv6vz\" (UniqueName: \"kubernetes.io/projected/d33a90f4-47d2-4445-b644-decbd1626039-kube-api-access-xv6vz\") pod \"error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm\" (UID: \"d33a90f4-47d2-4445-b644-decbd1626039\") " pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" Oct 30 22:29:02.914797 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:02.914766 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" Oct 30 22:29:03.205765 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:03.205738 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm"] Oct 30 22:29:03.206729 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:29:03.206703 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd33a90f4_47d2_4445_b644_decbd1626039.slice/crio-0f54ad300f2294365f8050a85caf69f40e97efc1386bf8126cadc577114dd76e WatchSource:0}: Error finding container 0f54ad300f2294365f8050a85caf69f40e97efc1386bf8126cadc577114dd76e: Status 404 returned error can't find the container with id 0f54ad300f2294365f8050a85caf69f40e97efc1386bf8126cadc577114dd76e Oct 30 22:29:03.452677 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:03.452641 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" event={"ID":"d33a90f4-47d2-4445-b644-decbd1626039","Type":"ContainerStarted","Data":"8bcfeaa51d9583176212ea3ebbf0a0165260c4c2332c4136e3101137427c5a0e"} Oct 30 22:29:03.453118 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:03.452695 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" event={"ID":"d33a90f4-47d2-4445-b644-decbd1626039","Type":"ContainerStarted","Data":"eaa25119a2950c83e853dd01e8f9f8c48f0fe5ae274c676acf5a0b8bf047f71d"} Oct 30 22:29:03.453118 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:03.452718 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" event={"ID":"d33a90f4-47d2-4445-b644-decbd1626039","Type":"ContainerStarted","Data":"0f54ad300f2294365f8050a85caf69f40e97efc1386bf8126cadc577114dd76e"} Oct 30 22:29:03.453118 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:03.452837 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" Oct 30 22:29:03.453118 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:03.452874 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" Oct 30 22:29:03.454037 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:03.453991 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" podUID="d33a90f4-47d2-4445-b644-decbd1626039" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused" Oct 30 22:29:03.493194 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:03.493147 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" podStartSLOduration=1.493134703 podStartE2EDuration="1.493134703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:29:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:29:03.492444603 +0000 UTC m=+1260.408130905" watchObservedRunningTime="2026-10-30 22:29:03.493134703 +0000 UTC m=+1260.408821004" Oct 30 22:29:04.456792 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:04.456757 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" podUID="d33a90f4-47d2-4445-b644-decbd1626039" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused" Oct 30 22:29:09.461179 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:09.461146 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" Oct 30 22:29:09.461757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:09.461729 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" podUID="d33a90f4-47d2-4445-b644-decbd1626039" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused" Oct 30 22:29:19.461962 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:19.461925 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" podUID="d33a90f4-47d2-4445-b644-decbd1626039" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused" Oct 30 22:29:29.462064 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:29.462007 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" podUID="d33a90f4-47d2-4445-b644-decbd1626039" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused" Oct 30 22:29:29.762720 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:29:29.762640 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:29:39.461896 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:39.461853 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" podUID="d33a90f4-47d2-4445-b644-decbd1626039" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused" Oct 30 22:29:39.846876 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:39.846793 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64"] Oct 30 22:29:39.847147 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:39.847101 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" podUID="aa9e10f4-f664-4910-bffe-7a9783520fe8" containerName="kserve-container" containerID="cri-o://2ee6b98c5058b3bb393c6501097ad8189ece03716611dec2cb9514cc7ce31859" gracePeriod=30 Oct 30 22:29:39.847206 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:39.847124 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" podUID="aa9e10f4-f664-4910-bffe-7a9783520fe8" containerName="kube-rbac-proxy" containerID="cri-o://75c0022995fb2b28bb75a782b8fb15d2e5a1a5efd9871b85e7a542bfbbb43cbb" gracePeriod=30 Oct 30 22:29:39.917427 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:39.917394 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:29:40.038436 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:40.038403 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:29:40.602915 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:40.602882 4615 generic.go:356] "Generic (PLEG): container finished" podID="aa9e10f4-f664-4910-bffe-7a9783520fe8" containerID="75c0022995fb2b28bb75a782b8fb15d2e5a1a5efd9871b85e7a542bfbbb43cbb" exitCode=0 Oct 30 22:29:40.603302 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:40.602955 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" event={"ID":"aa9e10f4-f664-4910-bffe-7a9783520fe8","Type":"ContainerDied","Data":"75c0022995fb2b28bb75a782b8fb15d2e5a1a5efd9871b85e7a542bfbbb43cbb"} Oct 30 22:29:41.313237 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:41.313198 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" podUID="aa9e10f4-f664-4910-bffe-7a9783520fe8" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.31:8643/healthz\": dial tcp 10.134.0.31:8643: connect: connection refused" Oct 30 22:29:41.317568 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:41.317543 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" podUID="aa9e10f4-f664-4910-bffe-7a9783520fe8" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused" Oct 30 22:29:43.088678 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:43.088655 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" Oct 30 22:29:43.249896 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:43.249864 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa9e10f4-f664-4910-bffe-7a9783520fe8-kube-api-access-thgbb\" (UniqueName: \"kubernetes.io/projected/aa9e10f4-f664-4910-bffe-7a9783520fe8-kube-api-access-thgbb\") pod \"aa9e10f4-f664-4910-bffe-7a9783520fe8\" (UID: \"aa9e10f4-f664-4910-bffe-7a9783520fe8\") " Oct 30 22:29:43.249896 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:43.249900 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aa9e10f4-f664-4910-bffe-7a9783520fe8-proxy-tls\" (UniqueName: \"kubernetes.io/secret/aa9e10f4-f664-4910-bffe-7a9783520fe8-proxy-tls\") pod \"aa9e10f4-f664-4910-bffe-7a9783520fe8\" (UID: \"aa9e10f4-f664-4910-bffe-7a9783520fe8\") " Oct 30 22:29:43.250120 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:43.249964 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aa9e10f4-f664-4910-bffe-7a9783520fe8-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aa9e10f4-f664-4910-bffe-7a9783520fe8-kube-rbac-proxy-sar-config\") pod \"aa9e10f4-f664-4910-bffe-7a9783520fe8\" (UID: \"aa9e10f4-f664-4910-bffe-7a9783520fe8\") " Oct 30 22:29:43.250372 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:43.250348 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa9e10f4-f664-4910-bffe-7a9783520fe8-kube-rbac-proxy-sar-config" pod "aa9e10f4-f664-4910-bffe-7a9783520fe8" (UID: "aa9e10f4-f664-4910-bffe-7a9783520fe8"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:29:43.251819 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:43.251797 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa9e10f4-f664-4910-bffe-7a9783520fe8-kube-api-access-thgbb" pod "aa9e10f4-f664-4910-bffe-7a9783520fe8" (UID: "aa9e10f4-f664-4910-bffe-7a9783520fe8"). InnerVolumeSpecName "kube-api-access-thgbb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:29:43.251912 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:43.251893 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa9e10f4-f664-4910-bffe-7a9783520fe8-proxy-tls" pod "aa9e10f4-f664-4910-bffe-7a9783520fe8" (UID: "aa9e10f4-f664-4910-bffe-7a9783520fe8"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:29:43.351382 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:43.351354 4615 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aa9e10f4-f664-4910-bffe-7a9783520fe8-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:29:43.351382 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:43.351382 4615 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thgbb\" (UniqueName: \"kubernetes.io/projected/aa9e10f4-f664-4910-bffe-7a9783520fe8-kube-api-access-thgbb\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:29:43.351535 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:43.351396 4615 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aa9e10f4-f664-4910-bffe-7a9783520fe8-proxy-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:29:43.616936 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:43.616851 4615 generic.go:356] "Generic (PLEG): container finished" podID="aa9e10f4-f664-4910-bffe-7a9783520fe8" containerID="2ee6b98c5058b3bb393c6501097ad8189ece03716611dec2cb9514cc7ce31859" exitCode=0 Oct 30 22:29:43.616936 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:43.616893 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" event={"ID":"aa9e10f4-f664-4910-bffe-7a9783520fe8","Type":"ContainerDied","Data":"2ee6b98c5058b3bb393c6501097ad8189ece03716611dec2cb9514cc7ce31859"} Oct 30 22:29:43.616936 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:43.616914 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" Oct 30 22:29:43.617220 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:43.616939 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64" event={"ID":"aa9e10f4-f664-4910-bffe-7a9783520fe8","Type":"ContainerDied","Data":"f49dc17b44810fc338299ac727eef4ea93cf2f6cd143ccb070e0a480a5b7cd34"} Oct 30 22:29:43.617220 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:43.616964 4615 scope.go:122] "RemoveContainer" containerID="75c0022995fb2b28bb75a782b8fb15d2e5a1a5efd9871b85e7a542bfbbb43cbb" Oct 30 22:29:43.624985 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:43.624967 4615 scope.go:122] "RemoveContainer" containerID="2ee6b98c5058b3bb393c6501097ad8189ece03716611dec2cb9514cc7ce31859" Oct 30 22:29:43.632038 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:43.632022 4615 scope.go:122] "RemoveContainer" containerID="75c0022995fb2b28bb75a782b8fb15d2e5a1a5efd9871b85e7a542bfbbb43cbb" Oct 30 22:29:43.632535 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:29:43.632513 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75c0022995fb2b28bb75a782b8fb15d2e5a1a5efd9871b85e7a542bfbbb43cbb\": container with ID starting with 75c0022995fb2b28bb75a782b8fb15d2e5a1a5efd9871b85e7a542bfbbb43cbb not found: ID does not exist" containerID="75c0022995fb2b28bb75a782b8fb15d2e5a1a5efd9871b85e7a542bfbbb43cbb" Oct 30 22:29:43.632624 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:43.632544 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75c0022995fb2b28bb75a782b8fb15d2e5a1a5efd9871b85e7a542bfbbb43cbb"} err="failed to get container status \"75c0022995fb2b28bb75a782b8fb15d2e5a1a5efd9871b85e7a542bfbbb43cbb\": rpc error: code = NotFound desc = could not find container \"75c0022995fb2b28bb75a782b8fb15d2e5a1a5efd9871b85e7a542bfbbb43cbb\": container with ID starting with 75c0022995fb2b28bb75a782b8fb15d2e5a1a5efd9871b85e7a542bfbbb43cbb not found: ID does not exist" Oct 30 22:29:43.632624 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:43.632559 4615 scope.go:122] "RemoveContainer" containerID="2ee6b98c5058b3bb393c6501097ad8189ece03716611dec2cb9514cc7ce31859" Oct 30 22:29:43.632807 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:29:43.632788 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ee6b98c5058b3bb393c6501097ad8189ece03716611dec2cb9514cc7ce31859\": container with ID starting with 2ee6b98c5058b3bb393c6501097ad8189ece03716611dec2cb9514cc7ce31859 not found: ID does not exist" containerID="2ee6b98c5058b3bb393c6501097ad8189ece03716611dec2cb9514cc7ce31859" Oct 30 22:29:43.632868 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:43.632817 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ee6b98c5058b3bb393c6501097ad8189ece03716611dec2cb9514cc7ce31859"} err="failed to get container status \"2ee6b98c5058b3bb393c6501097ad8189ece03716611dec2cb9514cc7ce31859\": rpc error: code = NotFound desc = could not find container \"2ee6b98c5058b3bb393c6501097ad8189ece03716611dec2cb9514cc7ce31859\": container with ID starting with 2ee6b98c5058b3bb393c6501097ad8189ece03716611dec2cb9514cc7ce31859 not found: ID does not exist" Oct 30 22:29:43.667884 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:43.667864 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64"] Oct 30 22:29:43.680346 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:43.680319 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-94a5a-predictor-9c964b699-9jp64"] Oct 30 22:29:45.232936 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:45.232903 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa9e10f4-f664-4910-bffe-7a9783520fe8" path="/var/lib/kubelet/pods/aa9e10f4-f664-4910-bffe-7a9783520fe8/volumes" Oct 30 22:29:47.265441 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:47.265370 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:29:47.382448 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:47.382411 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:29:47.960109 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:47.960074 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:29:48.016270 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.016238 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p"] Oct 30 22:29:48.016607 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.016594 4615 cpu_manager.go:414] "containerMap: removing container" podUID="aa9e10f4-f664-4910-bffe-7a9783520fe8" containerName="kube-rbac-proxy" Oct 30 22:29:48.016646 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.016609 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa9e10f4-f664-4910-bffe-7a9783520fe8" containerName="kube-rbac-proxy" Oct 30 22:29:48.016646 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.016623 4615 cpu_manager.go:414] "containerMap: removing container" podUID="aa9e10f4-f664-4910-bffe-7a9783520fe8" containerName="kserve-container" Oct 30 22:29:48.016646 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.016632 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa9e10f4-f664-4910-bffe-7a9783520fe8" containerName="kserve-container" Oct 30 22:29:48.016742 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.016690 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa9e10f4-f664-4910-bffe-7a9783520fe8" containerName="kube-rbac-proxy" Oct 30 22:29:48.016742 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.016700 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa9e10f4-f664-4910-bffe-7a9783520fe8" containerName="kserve-container" Oct 30 22:29:48.021152 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.021132 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" Oct 30 22:29:48.050480 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.050452 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1b4518b221\"/\"kube-root-ca.crt\"" Oct 30 22:29:48.050480 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.050469 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1b4518b221\"/\"error-404-isvc-4ace4-kube-rbac-proxy-sar-config\"" Oct 30 22:29:48.050650 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.050526 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-1b4518b221\"/\"error-404-isvc-4ace4-predictor-serving-cert\"" Oct 30 22:29:48.051619 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.051600 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p"] Oct 30 22:29:48.068809 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.068783 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:29:48.188430 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.188398 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fc1a2448-31b9-4353-b282-5d845377d93e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p\" (UID: \"fc1a2448-31b9-4353-b282-5d845377d93e\") " pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" Oct 30 22:29:48.188430 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.188434 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fc1a2448-31b9-4353-b282-5d845377d93e-proxy-tls\") pod \"error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p\" (UID: \"fc1a2448-31b9-4353-b282-5d845377d93e\") " pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" Oct 30 22:29:48.188657 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.188472 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8ptj\" (UniqueName: \"kubernetes.io/projected/fc1a2448-31b9-4353-b282-5d845377d93e-kube-api-access-s8ptj\") pod \"error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p\" (UID: \"fc1a2448-31b9-4353-b282-5d845377d93e\") " pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" Oct 30 22:29:48.289727 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.289650 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fc1a2448-31b9-4353-b282-5d845377d93e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p\" (UID: \"fc1a2448-31b9-4353-b282-5d845377d93e\") " pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" Oct 30 22:29:48.289727 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.289688 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fc1a2448-31b9-4353-b282-5d845377d93e-proxy-tls\") pod \"error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p\" (UID: \"fc1a2448-31b9-4353-b282-5d845377d93e\") " pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" Oct 30 22:29:48.290256 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.289732 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8ptj\" (UniqueName: \"kubernetes.io/projected/fc1a2448-31b9-4353-b282-5d845377d93e-kube-api-access-s8ptj\") pod \"error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p\" (UID: \"fc1a2448-31b9-4353-b282-5d845377d93e\") " pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" Oct 30 22:29:48.290544 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.290519 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fc1a2448-31b9-4353-b282-5d845377d93e-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p\" (UID: \"fc1a2448-31b9-4353-b282-5d845377d93e\") " pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" Oct 30 22:29:48.292087 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.292069 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fc1a2448-31b9-4353-b282-5d845377d93e-proxy-tls\") pod \"error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p\" (UID: \"fc1a2448-31b9-4353-b282-5d845377d93e\") " pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" Oct 30 22:29:48.309249 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.309221 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8ptj\" (UniqueName: \"kubernetes.io/projected/fc1a2448-31b9-4353-b282-5d845377d93e-kube-api-access-s8ptj\") pod \"error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p\" (UID: \"fc1a2448-31b9-4353-b282-5d845377d93e\") " pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" Oct 30 22:29:48.335104 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.335077 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" Oct 30 22:29:48.626531 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.626500 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p"] Oct 30 22:29:48.626866 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:29:48.626844 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc1a2448_31b9_4353_b282_5d845377d93e.slice/crio-45adf2bd1978616d00c293dbabbf15e01b994c438f48fe8caca2c7f6860e2309 WatchSource:0}: Error finding container 45adf2bd1978616d00c293dbabbf15e01b994c438f48fe8caca2c7f6860e2309: Status 404 returned error can't find the container with id 45adf2bd1978616d00c293dbabbf15e01b994c438f48fe8caca2c7f6860e2309 Oct 30 22:29:48.639753 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:48.639728 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" event={"ID":"fc1a2448-31b9-4353-b282-5d845377d93e","Type":"ContainerStarted","Data":"45adf2bd1978616d00c293dbabbf15e01b994c438f48fe8caca2c7f6860e2309"} Oct 30 22:29:49.462306 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:49.462265 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" podUID="d33a90f4-47d2-4445-b644-decbd1626039" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused" Oct 30 22:29:49.645831 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:49.645796 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" event={"ID":"fc1a2448-31b9-4353-b282-5d845377d93e","Type":"ContainerStarted","Data":"84e2b6ede621237ee956071724e2ea60b2ce6d643aeed3b1f744ca43c05bd2bd"} Oct 30 22:29:49.646007 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:49.645861 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" event={"ID":"fc1a2448-31b9-4353-b282-5d845377d93e","Type":"ContainerStarted","Data":"80cfd079402bccaa178990a1b884ba90c1afeb0300e7b0e65962c97be4e5f697"} Oct 30 22:29:49.646007 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:49.645896 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" Oct 30 22:29:49.646007 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:49.645921 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" Oct 30 22:29:49.647414 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:49.647386 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" podUID="fc1a2448-31b9-4353-b282-5d845377d93e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused" Oct 30 22:29:49.687233 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:49.687181 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" podStartSLOduration=2.68716904 podStartE2EDuration="2.68716904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:29:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:29:49.685396259 +0000 UTC m=+1306.601082562" watchObservedRunningTime="2026-10-30 22:29:49.68716904 +0000 UTC m=+1306.602855341" Oct 30 22:29:50.649739 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:50.649701 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" podUID="fc1a2448-31b9-4353-b282-5d845377d93e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused" Oct 30 22:29:50.840061 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:50.840021 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:29:54.970727 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:54.970691 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:29:55.654791 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:55.654765 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" Oct 30 22:29:55.655231 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:55.655206 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" podUID="fc1a2448-31b9-4353-b282-5d845377d93e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused" Oct 30 22:29:59.463097 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:29:59.463064 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" Oct 30 22:29:59.761714 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:29:59.761639 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:30:05.655747 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:05.655710 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" podUID="fc1a2448-31b9-4353-b282-5d845377d93e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused" Oct 30 22:30:12.320541 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:12.320503 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq"] Oct 30 22:30:12.325261 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:12.325243 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" Oct 30 22:30:12.342211 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:12.342190 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-cc765036c8\"/\"ensemble-graph-c1ed1-serving-cert\"" Oct 30 22:30:12.342333 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:12.342194 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-cc765036c8\"/\"ensemble-graph-c1ed1-kube-rbac-proxy-sar-config\"" Oct 30 22:30:12.345327 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:12.345303 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:30:12.355535 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:12.355511 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq"] Oct 30 22:30:12.365462 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:12.365435 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1bf494db-f7a0-4126-96aa-3819315dc2a0-proxy-tls\") pod \"ensemble-graph-c1ed1-75cfc869c7-mgkzq\" (UID: \"1bf494db-f7a0-4126-96aa-3819315dc2a0\") " pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" Oct 30 22:30:12.365548 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:12.365474 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bf494db-f7a0-4126-96aa-3819315dc2a0-openshift-service-ca-bundle\") pod \"ensemble-graph-c1ed1-75cfc869c7-mgkzq\" (UID: \"1bf494db-f7a0-4126-96aa-3819315dc2a0\") " pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" Oct 30 22:30:12.466402 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:12.466368 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1bf494db-f7a0-4126-96aa-3819315dc2a0-proxy-tls\") pod \"ensemble-graph-c1ed1-75cfc869c7-mgkzq\" (UID: \"1bf494db-f7a0-4126-96aa-3819315dc2a0\") " pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" Oct 30 22:30:12.466561 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:12.466415 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bf494db-f7a0-4126-96aa-3819315dc2a0-openshift-service-ca-bundle\") pod \"ensemble-graph-c1ed1-75cfc869c7-mgkzq\" (UID: \"1bf494db-f7a0-4126-96aa-3819315dc2a0\") " pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" Oct 30 22:30:12.467125 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:12.467105 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bf494db-f7a0-4126-96aa-3819315dc2a0-openshift-service-ca-bundle\") pod \"ensemble-graph-c1ed1-75cfc869c7-mgkzq\" (UID: \"1bf494db-f7a0-4126-96aa-3819315dc2a0\") " pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" Oct 30 22:30:12.468769 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:12.468748 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1bf494db-f7a0-4126-96aa-3819315dc2a0-proxy-tls\") pod \"ensemble-graph-c1ed1-75cfc869c7-mgkzq\" (UID: \"1bf494db-f7a0-4126-96aa-3819315dc2a0\") " pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" Oct 30 22:30:12.563260 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:12.563214 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:30:12.637871 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:12.637845 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" Oct 30 22:30:12.928056 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:12.928015 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq"] Oct 30 22:30:12.928966 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:30:12.928940 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bf494db_f7a0_4126_96aa_3819315dc2a0.slice/crio-e71342779312362690842f5cbda354451129fc5c97efb171cb07bf330fe33b21 WatchSource:0}: Error finding container e71342779312362690842f5cbda354451129fc5c97efb171cb07bf330fe33b21: Status 404 returned error can't find the container with id e71342779312362690842f5cbda354451129fc5c97efb171cb07bf330fe33b21 Oct 30 22:30:13.743702 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:13.743664 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" event={"ID":"1bf494db-f7a0-4126-96aa-3819315dc2a0","Type":"ContainerStarted","Data":"a51d3361cd5c4863518ca90e512b473014c7632f2001066e407a4f0d79102de5"} Oct 30 22:30:13.744133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:13.743721 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" event={"ID":"1bf494db-f7a0-4126-96aa-3819315dc2a0","Type":"ContainerStarted","Data":"e71342779312362690842f5cbda354451129fc5c97efb171cb07bf330fe33b21"} Oct 30 22:30:13.744133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:13.743820 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" Oct 30 22:30:13.784194 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:13.784150 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" podStartSLOduration=1.7841366349999999 podStartE2EDuration="1.784136635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:30:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:30:13.783443569 +0000 UTC m=+1330.699129875" watchObservedRunningTime="2026-10-30 22:30:13.784136635 +0000 UTC m=+1330.699822935" Oct 30 22:30:15.655811 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:15.655766 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" podUID="fc1a2448-31b9-4353-b282-5d845377d93e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused" Oct 30 22:30:19.754003 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:19.753972 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" Oct 30 22:30:23.400279 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:23.400248 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm"] Oct 30 22:30:23.400700 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:23.400461 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" podUID="d33a90f4-47d2-4445-b644-decbd1626039" containerName="kserve-container" containerID="cri-o://eaa25119a2950c83e853dd01e8f9f8c48f0fe5ae274c676acf5a0b8bf047f71d" gracePeriod=30 Oct 30 22:30:23.400700 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:23.400517 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" podUID="d33a90f4-47d2-4445-b644-decbd1626039" containerName="kube-rbac-proxy" containerID="cri-o://8bcfeaa51d9583176212ea3ebbf0a0165260c4c2332c4136e3101137427c5a0e" gracePeriod=30 Oct 30 22:30:23.453905 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:23.453870 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:30:23.594417 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:23.594378 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:30:23.787367 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:23.787287 4615 generic.go:356] "Generic (PLEG): container finished" podID="d33a90f4-47d2-4445-b644-decbd1626039" containerID="8bcfeaa51d9583176212ea3ebbf0a0165260c4c2332c4136e3101137427c5a0e" exitCode=0 Oct 30 22:30:23.787516 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:23.787362 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" event={"ID":"d33a90f4-47d2-4445-b644-decbd1626039","Type":"ContainerDied","Data":"8bcfeaa51d9583176212ea3ebbf0a0165260c4c2332c4136e3101137427c5a0e"} Oct 30 22:30:24.457394 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:24.457355 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" podUID="d33a90f4-47d2-4445-b644-decbd1626039" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.33:8643/healthz\": dial tcp 10.134.0.33:8643: connect: connection refused" Oct 30 22:30:25.655225 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:25.655187 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" podUID="fc1a2448-31b9-4353-b282-5d845377d93e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused" Oct 30 22:30:26.802924 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:26.802895 4615 generic.go:356] "Generic (PLEG): container finished" podID="d33a90f4-47d2-4445-b644-decbd1626039" containerID="eaa25119a2950c83e853dd01e8f9f8c48f0fe5ae274c676acf5a0b8bf047f71d" exitCode=0 Oct 30 22:30:26.803267 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:26.802973 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" event={"ID":"d33a90f4-47d2-4445-b644-decbd1626039","Type":"ContainerDied","Data":"eaa25119a2950c83e853dd01e8f9f8c48f0fe5ae274c676acf5a0b8bf047f71d"} Oct 30 22:30:26.848206 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:26.848184 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" Oct 30 22:30:26.882381 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:26.882361 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d33a90f4-47d2-4445-b644-decbd1626039-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d33a90f4-47d2-4445-b644-decbd1626039-kube-rbac-proxy-sar-config\") pod \"d33a90f4-47d2-4445-b644-decbd1626039\" (UID: \"d33a90f4-47d2-4445-b644-decbd1626039\") " Oct 30 22:30:26.882459 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:26.882399 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d33a90f4-47d2-4445-b644-decbd1626039-proxy-tls\" (UniqueName: \"kubernetes.io/secret/d33a90f4-47d2-4445-b644-decbd1626039-proxy-tls\") pod \"d33a90f4-47d2-4445-b644-decbd1626039\" (UID: \"d33a90f4-47d2-4445-b644-decbd1626039\") " Oct 30 22:30:26.882459 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:26.882444 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d33a90f4-47d2-4445-b644-decbd1626039-kube-api-access-xv6vz\" (UniqueName: \"kubernetes.io/projected/d33a90f4-47d2-4445-b644-decbd1626039-kube-api-access-xv6vz\") pod \"d33a90f4-47d2-4445-b644-decbd1626039\" (UID: \"d33a90f4-47d2-4445-b644-decbd1626039\") " Oct 30 22:30:26.882727 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:26.882701 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d33a90f4-47d2-4445-b644-decbd1626039-kube-rbac-proxy-sar-config" pod "d33a90f4-47d2-4445-b644-decbd1626039" (UID: "d33a90f4-47d2-4445-b644-decbd1626039"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:30:26.884393 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:26.884369 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d33a90f4-47d2-4445-b644-decbd1626039-proxy-tls" pod "d33a90f4-47d2-4445-b644-decbd1626039" (UID: "d33a90f4-47d2-4445-b644-decbd1626039"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:30:26.884483 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:26.884432 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d33a90f4-47d2-4445-b644-decbd1626039-kube-api-access-xv6vz" pod "d33a90f4-47d2-4445-b644-decbd1626039" (UID: "d33a90f4-47d2-4445-b644-decbd1626039"). InnerVolumeSpecName "kube-api-access-xv6vz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:30:26.984157 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:26.984084 4615 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d33a90f4-47d2-4445-b644-decbd1626039-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:30:26.984157 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:26.984109 4615 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d33a90f4-47d2-4445-b644-decbd1626039-proxy-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:30:26.984157 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:26.984121 4615 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xv6vz\" (UniqueName: \"kubernetes.io/projected/d33a90f4-47d2-4445-b644-decbd1626039-kube-api-access-xv6vz\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:30:27.809246 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:27.809214 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" Oct 30 22:30:27.809246 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:27.809231 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm" event={"ID":"d33a90f4-47d2-4445-b644-decbd1626039","Type":"ContainerDied","Data":"0f54ad300f2294365f8050a85caf69f40e97efc1386bf8126cadc577114dd76e"} Oct 30 22:30:27.809782 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:27.809293 4615 scope.go:122] "RemoveContainer" containerID="8bcfeaa51d9583176212ea3ebbf0a0165260c4c2332c4136e3101137427c5a0e" Oct 30 22:30:27.816839 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:27.816822 4615 scope.go:122] "RemoveContainer" containerID="eaa25119a2950c83e853dd01e8f9f8c48f0fe5ae274c676acf5a0b8bf047f71d" Oct 30 22:30:27.861466 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:27.861435 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm"] Oct 30 22:30:27.873857 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:27.873831 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-c1ed1-predictor-8654c76d8-xmpqm"] Oct 30 22:30:28.804764 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:28.804712 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:30:28.969555 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:28.969511 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:30:29.210809 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.210779 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:30:29.232511 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.232486 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d33a90f4-47d2-4445-b644-decbd1626039" path="/var/lib/kubelet/pods/d33a90f4-47d2-4445-b644-decbd1626039/volumes" Oct 30 22:30:29.243036 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.243015 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j"] Oct 30 22:30:29.243351 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.243338 4615 cpu_manager.go:414] "containerMap: removing container" podUID="d33a90f4-47d2-4445-b644-decbd1626039" containerName="kube-rbac-proxy" Oct 30 22:30:29.243394 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.243352 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d33a90f4-47d2-4445-b644-decbd1626039" containerName="kube-rbac-proxy" Oct 30 22:30:29.243394 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.243370 4615 cpu_manager.go:414] "containerMap: removing container" podUID="d33a90f4-47d2-4445-b644-decbd1626039" containerName="kserve-container" Oct 30 22:30:29.243394 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.243374 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d33a90f4-47d2-4445-b644-decbd1626039" containerName="kserve-container" Oct 30 22:30:29.243497 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.243425 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="d33a90f4-47d2-4445-b644-decbd1626039" containerName="kube-rbac-proxy" Oct 30 22:30:29.243497 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.243434 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="d33a90f4-47d2-4445-b644-decbd1626039" containerName="kserve-container" Oct 30 22:30:29.247918 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.247904 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" Oct 30 22:30:29.276251 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.276229 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-cc765036c8\"/\"error-404-isvc-83e67-kube-rbac-proxy-sar-config\"" Oct 30 22:30:29.276366 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.276250 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-cc765036c8\"/\"kube-root-ca.crt\"" Oct 30 22:30:29.276366 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.276327 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-cc765036c8\"/\"error-404-isvc-83e67-predictor-serving-cert\"" Oct 30 22:30:29.277507 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.277488 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j"] Oct 30 22:30:29.303087 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.303060 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq74j\" (UniqueName: \"kubernetes.io/projected/7c365bd6-8878-43b6-b942-4098b4df006b-kube-api-access-vq74j\") pod \"error-404-isvc-83e67-predictor-767594485d-svp5j\" (UID: \"7c365bd6-8878-43b6-b942-4098b4df006b\") " pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" Oct 30 22:30:29.303187 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.303129 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7c365bd6-8878-43b6-b942-4098b4df006b-proxy-tls\") pod \"error-404-isvc-83e67-predictor-767594485d-svp5j\" (UID: \"7c365bd6-8878-43b6-b942-4098b4df006b\") " pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" Oct 30 22:30:29.303187 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.303159 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7c365bd6-8878-43b6-b942-4098b4df006b-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-83e67-predictor-767594485d-svp5j\" (UID: \"7c365bd6-8878-43b6-b942-4098b4df006b\") " pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" Oct 30 22:30:29.375804 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.375768 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:30:29.403937 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.403913 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq74j\" (UniqueName: \"kubernetes.io/projected/7c365bd6-8878-43b6-b942-4098b4df006b-kube-api-access-vq74j\") pod \"error-404-isvc-83e67-predictor-767594485d-svp5j\" (UID: \"7c365bd6-8878-43b6-b942-4098b4df006b\") " pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" Oct 30 22:30:29.404017 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.403959 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7c365bd6-8878-43b6-b942-4098b4df006b-proxy-tls\") pod \"error-404-isvc-83e67-predictor-767594485d-svp5j\" (UID: \"7c365bd6-8878-43b6-b942-4098b4df006b\") " pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" Oct 30 22:30:29.404017 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.403978 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7c365bd6-8878-43b6-b942-4098b4df006b-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-83e67-predictor-767594485d-svp5j\" (UID: \"7c365bd6-8878-43b6-b942-4098b4df006b\") " pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" Oct 30 22:30:29.404611 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.404593 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7c365bd6-8878-43b6-b942-4098b4df006b-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-83e67-predictor-767594485d-svp5j\" (UID: \"7c365bd6-8878-43b6-b942-4098b4df006b\") " pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" Oct 30 22:30:29.406171 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.406152 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7c365bd6-8878-43b6-b942-4098b4df006b-proxy-tls\") pod \"error-404-isvc-83e67-predictor-767594485d-svp5j\" (UID: \"7c365bd6-8878-43b6-b942-4098b4df006b\") " pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" Oct 30 22:30:29.424412 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.424387 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq74j\" (UniqueName: \"kubernetes.io/projected/7c365bd6-8878-43b6-b942-4098b4df006b-kube-api-access-vq74j\") pod \"error-404-isvc-83e67-predictor-767594485d-svp5j\" (UID: \"7c365bd6-8878-43b6-b942-4098b4df006b\") " pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" Oct 30 22:30:29.561827 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.561755 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" Oct 30 22:30:29.761739 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:30:29.761706 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:30:29.852326 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:29.852299 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j"] Oct 30 22:30:29.853613 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:30:29.853587 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c365bd6_8878_43b6_b942_4098b4df006b.slice/crio-848af9c804a908353fce8c1a97c2f4e4ce835c39e1c644473e5f313d3ae5a6d8 WatchSource:0}: Error finding container 848af9c804a908353fce8c1a97c2f4e4ce835c39e1c644473e5f313d3ae5a6d8: Status 404 returned error can't find the container with id 848af9c804a908353fce8c1a97c2f4e4ce835c39e1c644473e5f313d3ae5a6d8 Oct 30 22:30:30.823872 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:30.823840 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" event={"ID":"7c365bd6-8878-43b6-b942-4098b4df006b","Type":"ContainerStarted","Data":"3ed1b7767bfa8e35cc397fe68a61046900fdf9905997eb8585ed47f359db0299"} Oct 30 22:30:30.824265 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:30.823901 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" event={"ID":"7c365bd6-8878-43b6-b942-4098b4df006b","Type":"ContainerStarted","Data":"dc01a3da9b2c60992ce69cefb59a2c054d9d54640a4de817a5454e07a64fd223"} Oct 30 22:30:30.824265 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:30.823922 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" event={"ID":"7c365bd6-8878-43b6-b942-4098b4df006b","Type":"ContainerStarted","Data":"848af9c804a908353fce8c1a97c2f4e4ce835c39e1c644473e5f313d3ae5a6d8"} Oct 30 22:30:30.824265 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:30.823957 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" Oct 30 22:30:30.865015 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:30.864966 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" podStartSLOduration=1.864950917 podStartE2EDuration="1.864950917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:30:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:30:30.863962354 +0000 UTC m=+1347.779648655" watchObservedRunningTime="2026-10-30 22:30:30.864950917 +0000 UTC m=+1347.780637222" Oct 30 22:30:31.827740 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:31.827703 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" Oct 30 22:30:31.829123 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:31.829096 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" podUID="7c365bd6-8878-43b6-b942-4098b4df006b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused" Oct 30 22:30:32.831742 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:32.831707 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" podUID="7c365bd6-8878-43b6-b942-4098b4df006b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused" Oct 30 22:30:35.656003 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:35.655966 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" podUID="fc1a2448-31b9-4353-b282-5d845377d93e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused" Oct 30 22:30:37.836133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:37.836104 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" Oct 30 22:30:37.836620 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:37.836574 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" podUID="7c365bd6-8878-43b6-b942-4098b4df006b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused" Oct 30 22:30:45.656006 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:45.655979 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" Oct 30 22:30:47.837015 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:47.836975 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" podUID="7c365bd6-8878-43b6-b942-4098b4df006b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused" Oct 30 22:30:50.871229 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:50.871183 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:30:57.334122 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:57.334085 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh"] Oct 30 22:30:57.337687 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:57.337662 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" Oct 30 22:30:57.354629 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:57.354598 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:30:57.356336 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:57.356312 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh"] Oct 30 22:30:57.367216 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:57.367189 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1b4518b221\"/\"sequence-graph-4ace4-kube-rbac-proxy-sar-config\"" Oct 30 22:30:57.367315 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:57.367222 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-1b4518b221\"/\"sequence-graph-4ace4-serving-cert\"" Oct 30 22:30:57.441266 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:57.441231 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd617f4a-e247-485e-85c7-83e715540f25-openshift-service-ca-bundle\") pod \"sequence-graph-4ace4-86496f9c9-lqblh\" (UID: \"dd617f4a-e247-485e-85c7-83e715540f25\") " pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" Oct 30 22:30:57.441266 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:57.441264 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd617f4a-e247-485e-85c7-83e715540f25-proxy-tls\") pod \"sequence-graph-4ace4-86496f9c9-lqblh\" (UID: \"dd617f4a-e247-485e-85c7-83e715540f25\") " pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" Oct 30 22:30:57.470022 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:57.469990 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:30:57.542242 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:57.542215 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd617f4a-e247-485e-85c7-83e715540f25-openshift-service-ca-bundle\") pod \"sequence-graph-4ace4-86496f9c9-lqblh\" (UID: \"dd617f4a-e247-485e-85c7-83e715540f25\") " pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" Oct 30 22:30:57.542385 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:57.542248 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd617f4a-e247-485e-85c7-83e715540f25-proxy-tls\") pod \"sequence-graph-4ace4-86496f9c9-lqblh\" (UID: \"dd617f4a-e247-485e-85c7-83e715540f25\") " pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" Oct 30 22:30:57.542823 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:57.542805 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd617f4a-e247-485e-85c7-83e715540f25-openshift-service-ca-bundle\") pod \"sequence-graph-4ace4-86496f9c9-lqblh\" (UID: \"dd617f4a-e247-485e-85c7-83e715540f25\") " pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" Oct 30 22:30:57.544608 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:57.544587 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd617f4a-e247-485e-85c7-83e715540f25-proxy-tls\") pod \"sequence-graph-4ace4-86496f9c9-lqblh\" (UID: \"dd617f4a-e247-485e-85c7-83e715540f25\") " pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" Oct 30 22:30:57.650122 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:57.650077 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" Oct 30 22:30:57.836766 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:57.836732 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" podUID="7c365bd6-8878-43b6-b942-4098b4df006b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused" Oct 30 22:30:57.940479 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:57.940449 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh"] Oct 30 22:30:57.941278 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:30:57.941251 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd617f4a_e247_485e_85c7_83e715540f25.slice/crio-3f599bddfd0d5cf2e2e212312a29643755f33d8609963dfe84d9c25b409bdfd0 WatchSource:0}: Error finding container 3f599bddfd0d5cf2e2e212312a29643755f33d8609963dfe84d9c25b409bdfd0: Status 404 returned error can't find the container with id 3f599bddfd0d5cf2e2e212312a29643755f33d8609963dfe84d9c25b409bdfd0 Oct 30 22:30:58.946095 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:58.946034 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" event={"ID":"dd617f4a-e247-485e-85c7-83e715540f25","Type":"ContainerStarted","Data":"b8667b84ee8a2b5848996e446102f03d58392555d723f28dd53843774a60eec9"} Oct 30 22:30:58.946535 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:58.946110 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" event={"ID":"dd617f4a-e247-485e-85c7-83e715540f25","Type":"ContainerStarted","Data":"3f599bddfd0d5cf2e2e212312a29643755f33d8609963dfe84d9c25b409bdfd0"} Oct 30 22:30:58.946535 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:58.946133 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" Oct 30 22:30:58.986063 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:30:58.986007 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" podStartSLOduration=1.985994282 podStartE2EDuration="1.985994282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:30:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:30:58.985003078 +0000 UTC m=+1375.900689379" watchObservedRunningTime="2026-10-30 22:30:58.985994282 +0000 UTC m=+1375.901680586" Oct 30 22:30:59.763585 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:30:59.763542 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:31:00.298436 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:00.298386 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:31:01.053526 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:01.053494 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:31:04.956257 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:04.956228 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" Oct 30 22:31:07.837330 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:07.837291 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" podUID="7c365bd6-8878-43b6-b942-4098b4df006b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused" Oct 30 22:31:08.710476 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:08.710446 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p"] Oct 30 22:31:08.710730 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:08.710704 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" podUID="fc1a2448-31b9-4353-b282-5d845377d93e" containerName="kserve-container" containerID="cri-o://80cfd079402bccaa178990a1b884ba90c1afeb0300e7b0e65962c97be4e5f697" gracePeriod=30 Oct 30 22:31:08.710819 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:08.710719 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" podUID="fc1a2448-31b9-4353-b282-5d845377d93e" containerName="kube-rbac-proxy" containerID="cri-o://84e2b6ede621237ee956071724e2ea60b2ce6d643aeed3b1f744ca43c05bd2bd" gracePeriod=30 Oct 30 22:31:08.989451 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:08.989364 4615 generic.go:356] "Generic (PLEG): container finished" podID="fc1a2448-31b9-4353-b282-5d845377d93e" containerID="84e2b6ede621237ee956071724e2ea60b2ce6d643aeed3b1f744ca43c05bd2bd" exitCode=0 Oct 30 22:31:08.989451 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:08.989438 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" event={"ID":"fc1a2448-31b9-4353-b282-5d845377d93e","Type":"ContainerDied","Data":"84e2b6ede621237ee956071724e2ea60b2ce6d643aeed3b1f744ca43c05bd2bd"} Oct 30 22:31:10.168801 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:10.168770 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:31:10.287220 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:10.287191 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:31:10.650575 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:10.650533 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" podUID="fc1a2448-31b9-4353-b282-5d845377d93e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.34:8643/healthz\": dial tcp 10.134.0.34:8643: connect: connection refused" Oct 30 22:31:12.249200 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:12.249175 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" Oct 30 22:31:12.364216 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:12.364143 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc1a2448-31b9-4353-b282-5d845377d93e-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fc1a2448-31b9-4353-b282-5d845377d93e-kube-rbac-proxy-sar-config\") pod \"fc1a2448-31b9-4353-b282-5d845377d93e\" (UID: \"fc1a2448-31b9-4353-b282-5d845377d93e\") " Oct 30 22:31:12.364216 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:12.364183 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc1a2448-31b9-4353-b282-5d845377d93e-proxy-tls\" (UniqueName: \"kubernetes.io/secret/fc1a2448-31b9-4353-b282-5d845377d93e-proxy-tls\") pod \"fc1a2448-31b9-4353-b282-5d845377d93e\" (UID: \"fc1a2448-31b9-4353-b282-5d845377d93e\") " Oct 30 22:31:12.364216 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:12.364211 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc1a2448-31b9-4353-b282-5d845377d93e-kube-api-access-s8ptj\" (UniqueName: \"kubernetes.io/projected/fc1a2448-31b9-4353-b282-5d845377d93e-kube-api-access-s8ptj\") pod \"fc1a2448-31b9-4353-b282-5d845377d93e\" (UID: \"fc1a2448-31b9-4353-b282-5d845377d93e\") " Oct 30 22:31:12.364626 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:12.364593 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc1a2448-31b9-4353-b282-5d845377d93e-kube-rbac-proxy-sar-config" pod "fc1a2448-31b9-4353-b282-5d845377d93e" (UID: "fc1a2448-31b9-4353-b282-5d845377d93e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:31:12.366227 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:12.366208 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc1a2448-31b9-4353-b282-5d845377d93e-kube-api-access-s8ptj" pod "fc1a2448-31b9-4353-b282-5d845377d93e" (UID: "fc1a2448-31b9-4353-b282-5d845377d93e"). InnerVolumeSpecName "kube-api-access-s8ptj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:31:12.366303 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:12.366248 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc1a2448-31b9-4353-b282-5d845377d93e-proxy-tls" pod "fc1a2448-31b9-4353-b282-5d845377d93e" (UID: "fc1a2448-31b9-4353-b282-5d845377d93e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:31:12.465718 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:12.465686 4615 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/fc1a2448-31b9-4353-b282-5d845377d93e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:31:12.465718 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:12.465714 4615 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/fc1a2448-31b9-4353-b282-5d845377d93e-proxy-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:31:12.465718 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:12.465726 4615 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8ptj\" (UniqueName: \"kubernetes.io/projected/fc1a2448-31b9-4353-b282-5d845377d93e-kube-api-access-s8ptj\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:31:13.007065 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:13.007023 4615 generic.go:356] "Generic (PLEG): container finished" podID="fc1a2448-31b9-4353-b282-5d845377d93e" containerID="80cfd079402bccaa178990a1b884ba90c1afeb0300e7b0e65962c97be4e5f697" exitCode=0 Oct 30 22:31:13.007237 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:13.007084 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" event={"ID":"fc1a2448-31b9-4353-b282-5d845377d93e","Type":"ContainerDied","Data":"80cfd079402bccaa178990a1b884ba90c1afeb0300e7b0e65962c97be4e5f697"} Oct 30 22:31:13.007237 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:13.007100 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" Oct 30 22:31:13.007237 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:13.007155 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p" event={"ID":"fc1a2448-31b9-4353-b282-5d845377d93e","Type":"ContainerDied","Data":"45adf2bd1978616d00c293dbabbf15e01b994c438f48fe8caca2c7f6860e2309"} Oct 30 22:31:13.007237 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:13.007181 4615 scope.go:122] "RemoveContainer" containerID="84e2b6ede621237ee956071724e2ea60b2ce6d643aeed3b1f744ca43c05bd2bd" Oct 30 22:31:13.015323 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:13.015307 4615 scope.go:122] "RemoveContainer" containerID="80cfd079402bccaa178990a1b884ba90c1afeb0300e7b0e65962c97be4e5f697" Oct 30 22:31:13.022446 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:13.022428 4615 scope.go:122] "RemoveContainer" containerID="84e2b6ede621237ee956071724e2ea60b2ce6d643aeed3b1f744ca43c05bd2bd" Oct 30 22:31:13.022673 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:31:13.022654 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84e2b6ede621237ee956071724e2ea60b2ce6d643aeed3b1f744ca43c05bd2bd\": container with ID starting with 84e2b6ede621237ee956071724e2ea60b2ce6d643aeed3b1f744ca43c05bd2bd not found: ID does not exist" containerID="84e2b6ede621237ee956071724e2ea60b2ce6d643aeed3b1f744ca43c05bd2bd" Oct 30 22:31:13.022736 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:13.022680 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84e2b6ede621237ee956071724e2ea60b2ce6d643aeed3b1f744ca43c05bd2bd"} err="failed to get container status \"84e2b6ede621237ee956071724e2ea60b2ce6d643aeed3b1f744ca43c05bd2bd\": rpc error: code = NotFound desc = could not find container \"84e2b6ede621237ee956071724e2ea60b2ce6d643aeed3b1f744ca43c05bd2bd\": container with ID starting with 84e2b6ede621237ee956071724e2ea60b2ce6d643aeed3b1f744ca43c05bd2bd not found: ID does not exist" Oct 30 22:31:13.022736 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:13.022696 4615 scope.go:122] "RemoveContainer" containerID="80cfd079402bccaa178990a1b884ba90c1afeb0300e7b0e65962c97be4e5f697" Oct 30 22:31:13.022926 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:31:13.022908 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80cfd079402bccaa178990a1b884ba90c1afeb0300e7b0e65962c97be4e5f697\": container with ID starting with 80cfd079402bccaa178990a1b884ba90c1afeb0300e7b0e65962c97be4e5f697 not found: ID does not exist" containerID="80cfd079402bccaa178990a1b884ba90c1afeb0300e7b0e65962c97be4e5f697" Oct 30 22:31:13.022964 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:13.022932 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80cfd079402bccaa178990a1b884ba90c1afeb0300e7b0e65962c97be4e5f697"} err="failed to get container status \"80cfd079402bccaa178990a1b884ba90c1afeb0300e7b0e65962c97be4e5f697\": rpc error: code = NotFound desc = could not find container \"80cfd079402bccaa178990a1b884ba90c1afeb0300e7b0e65962c97be4e5f697\": container with ID starting with 80cfd079402bccaa178990a1b884ba90c1afeb0300e7b0e65962c97be4e5f697 not found: ID does not exist" Oct 30 22:31:13.058712 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:13.058682 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p"] Oct 30 22:31:13.071081 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:13.071056 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-4ace4-predictor-66d54bd8d5-pp28p"] Oct 30 22:31:13.233526 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:13.233494 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc1a2448-31b9-4353-b282-5d845377d93e" path="/var/lib/kubelet/pods/fc1a2448-31b9-4353-b282-5d845377d93e/volumes" Oct 30 22:31:14.127778 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.127744 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:31:14.248576 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.248535 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:31:14.541755 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.541718 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:31:14.584240 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.584213 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5"] Oct 30 22:31:14.584542 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.584530 4615 cpu_manager.go:414] "containerMap: removing container" podUID="fc1a2448-31b9-4353-b282-5d845377d93e" containerName="kserve-container" Oct 30 22:31:14.584588 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.584543 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc1a2448-31b9-4353-b282-5d845377d93e" containerName="kserve-container" Oct 30 22:31:14.584588 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.584556 4615 cpu_manager.go:414] "containerMap: removing container" podUID="fc1a2448-31b9-4353-b282-5d845377d93e" containerName="kube-rbac-proxy" Oct 30 22:31:14.584588 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.584561 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc1a2448-31b9-4353-b282-5d845377d93e" containerName="kube-rbac-proxy" Oct 30 22:31:14.584680 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.584627 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc1a2448-31b9-4353-b282-5d845377d93e" containerName="kube-rbac-proxy" Oct 30 22:31:14.584680 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.584642 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc1a2448-31b9-4353-b282-5d845377d93e" containerName="kserve-container" Oct 30 22:31:14.589264 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.589248 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" Oct 30 22:31:14.605625 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.605599 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1b4518b221\"/\"error-404-isvc-15dee-kube-rbac-proxy-sar-config\"" Oct 30 22:31:14.605750 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.605685 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1b4518b221\"/\"kube-root-ca.crt\"" Oct 30 22:31:14.606895 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.606876 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5"] Oct 30 22:31:14.617751 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.617734 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-1b4518b221\"/\"error-404-isvc-15dee-predictor-serving-cert\"" Oct 30 22:31:14.654561 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.654541 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:31:14.684975 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.684955 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0a58038-763f-4cd6-8921-c073bf85cb81-proxy-tls\") pod \"error-404-isvc-15dee-predictor-66f695dc6c-fcvf5\" (UID: \"f0a58038-763f-4cd6-8921-c073bf85cb81\") " pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" Oct 30 22:31:14.685088 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.684983 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrnzt\" (UniqueName: \"kubernetes.io/projected/f0a58038-763f-4cd6-8921-c073bf85cb81-kube-api-access-jrnzt\") pod \"error-404-isvc-15dee-predictor-66f695dc6c-fcvf5\" (UID: \"f0a58038-763f-4cd6-8921-c073bf85cb81\") " pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" Oct 30 22:31:14.685088 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.685018 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0a58038-763f-4cd6-8921-c073bf85cb81-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-15dee-predictor-66f695dc6c-fcvf5\" (UID: \"f0a58038-763f-4cd6-8921-c073bf85cb81\") " pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" Oct 30 22:31:14.785711 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.785678 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0a58038-763f-4cd6-8921-c073bf85cb81-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-15dee-predictor-66f695dc6c-fcvf5\" (UID: \"f0a58038-763f-4cd6-8921-c073bf85cb81\") " pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" Oct 30 22:31:14.785858 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.785751 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0a58038-763f-4cd6-8921-c073bf85cb81-proxy-tls\") pod \"error-404-isvc-15dee-predictor-66f695dc6c-fcvf5\" (UID: \"f0a58038-763f-4cd6-8921-c073bf85cb81\") " pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" Oct 30 22:31:14.785858 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.785771 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrnzt\" (UniqueName: \"kubernetes.io/projected/f0a58038-763f-4cd6-8921-c073bf85cb81-kube-api-access-jrnzt\") pod \"error-404-isvc-15dee-predictor-66f695dc6c-fcvf5\" (UID: \"f0a58038-763f-4cd6-8921-c073bf85cb81\") " pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" Oct 30 22:31:14.786311 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.786291 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0a58038-763f-4cd6-8921-c073bf85cb81-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-15dee-predictor-66f695dc6c-fcvf5\" (UID: \"f0a58038-763f-4cd6-8921-c073bf85cb81\") " pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" Oct 30 22:31:14.788101 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.788078 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0a58038-763f-4cd6-8921-c073bf85cb81-proxy-tls\") pod \"error-404-isvc-15dee-predictor-66f695dc6c-fcvf5\" (UID: \"f0a58038-763f-4cd6-8921-c073bf85cb81\") " pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" Oct 30 22:31:14.805635 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.805586 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrnzt\" (UniqueName: \"kubernetes.io/projected/f0a58038-763f-4cd6-8921-c073bf85cb81-kube-api-access-jrnzt\") pod \"error-404-isvc-15dee-predictor-66f695dc6c-fcvf5\" (UID: \"f0a58038-763f-4cd6-8921-c073bf85cb81\") " pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" Oct 30 22:31:14.903440 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:14.903407 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" Oct 30 22:31:15.194907 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:15.194879 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5"] Oct 30 22:31:15.195587 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:31:15.195567 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0a58038_763f_4cd6_8921_c073bf85cb81.slice/crio-11db88d1eb5e402092d98157ef4bdbe43cadbf27615158e040101a5c40e022ea WatchSource:0}: Error finding container 11db88d1eb5e402092d98157ef4bdbe43cadbf27615158e040101a5c40e022ea: Status 404 returned error can't find the container with id 11db88d1eb5e402092d98157ef4bdbe43cadbf27615158e040101a5c40e022ea Oct 30 22:31:16.021922 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:16.021888 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" event={"ID":"f0a58038-763f-4cd6-8921-c073bf85cb81","Type":"ContainerStarted","Data":"9e62abcaaa0eb1782b12dcc94440f9690366a29328b74becd540e0c22e1e0c17"} Oct 30 22:31:16.022111 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:16.021942 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" event={"ID":"f0a58038-763f-4cd6-8921-c073bf85cb81","Type":"ContainerStarted","Data":"1621826ec6011fbb862e07be7ae7c672b82a7e8e135c42f8c9140030caeef230"} Oct 30 22:31:16.022111 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:16.021963 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" event={"ID":"f0a58038-763f-4cd6-8921-c073bf85cb81","Type":"ContainerStarted","Data":"11db88d1eb5e402092d98157ef4bdbe43cadbf27615158e040101a5c40e022ea"} Oct 30 22:31:16.022111 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:16.021987 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" Oct 30 22:31:16.063688 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:16.063642 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" podStartSLOduration=2.063629462 podStartE2EDuration="2.063629462s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:31:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:31:16.062348673 +0000 UTC m=+1392.978034974" watchObservedRunningTime="2026-10-30 22:31:16.063629462 +0000 UTC m=+1392.979315763" Oct 30 22:31:17.025824 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:17.025791 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" Oct 30 22:31:17.027109 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:17.027077 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" podUID="f0a58038-763f-4cd6-8921-c073bf85cb81" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused" Oct 30 22:31:17.837132 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:17.837089 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" podUID="7c365bd6-8878-43b6-b942-4098b4df006b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused" Oct 30 22:31:18.029727 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:18.029685 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" podUID="f0a58038-763f-4cd6-8921-c073bf85cb81" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused" Oct 30 22:31:23.034185 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:23.034151 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" Oct 30 22:31:23.034657 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:23.034629 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" podUID="f0a58038-763f-4cd6-8921-c073bf85cb81" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused" Oct 30 22:31:23.201291 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:23.201257 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:31:24.355821 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:24.355783 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:31:27.837913 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:27.837879 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" Oct 30 22:31:29.763014 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:31:29.762981 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:31:33.034693 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:33.034647 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" podUID="f0a58038-763f-4cd6-8921-c073bf85cb81" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused" Oct 30 22:31:38.914877 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:38.914846 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh"] Oct 30 22:31:38.918445 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:38.918426 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" Oct 30 22:31:38.928420 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:38.928397 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:31:38.938254 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:38.938227 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh"] Oct 30 22:31:38.948529 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:38.948499 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-cc765036c8\"/\"ensemble-graph-83e67-serving-cert\"" Oct 30 22:31:38.948638 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:38.948548 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-cc765036c8\"/\"ensemble-graph-83e67-kube-rbac-proxy-sar-config\"" Oct 30 22:31:38.998130 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:38.998101 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/876dbfb5-875f-4336-8247-982a6ac58d37-proxy-tls\") pod \"ensemble-graph-83e67-68c5d8576-j9hdh\" (UID: \"876dbfb5-875f-4336-8247-982a6ac58d37\") " pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" Oct 30 22:31:38.998271 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:38.998152 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/876dbfb5-875f-4336-8247-982a6ac58d37-openshift-service-ca-bundle\") pod \"ensemble-graph-83e67-68c5d8576-j9hdh\" (UID: \"876dbfb5-875f-4336-8247-982a6ac58d37\") " pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" Oct 30 22:31:39.084741 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:39.084715 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:31:39.099295 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:39.099268 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/876dbfb5-875f-4336-8247-982a6ac58d37-proxy-tls\") pod \"ensemble-graph-83e67-68c5d8576-j9hdh\" (UID: \"876dbfb5-875f-4336-8247-982a6ac58d37\") " pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" Oct 30 22:31:39.099390 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:39.099314 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/876dbfb5-875f-4336-8247-982a6ac58d37-openshift-service-ca-bundle\") pod \"ensemble-graph-83e67-68c5d8576-j9hdh\" (UID: \"876dbfb5-875f-4336-8247-982a6ac58d37\") " pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" Oct 30 22:31:39.099894 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:39.099870 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/876dbfb5-875f-4336-8247-982a6ac58d37-openshift-service-ca-bundle\") pod \"ensemble-graph-83e67-68c5d8576-j9hdh\" (UID: \"876dbfb5-875f-4336-8247-982a6ac58d37\") " pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" Oct 30 22:31:39.101655 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:39.101638 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/876dbfb5-875f-4336-8247-982a6ac58d37-proxy-tls\") pod \"ensemble-graph-83e67-68c5d8576-j9hdh\" (UID: \"876dbfb5-875f-4336-8247-982a6ac58d37\") " pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" Oct 30 22:31:39.231184 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:39.231108 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" Oct 30 22:31:39.519395 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:39.519368 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh"] Oct 30 22:31:39.520161 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:31:39.520131 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod876dbfb5_875f_4336_8247_982a6ac58d37.slice/crio-59a17a6bfe87d48c37435a19703acc1751bbf4501fad69a6da6530090ca52ef9 WatchSource:0}: Error finding container 59a17a6bfe87d48c37435a19703acc1751bbf4501fad69a6da6530090ca52ef9: Status 404 returned error can't find the container with id 59a17a6bfe87d48c37435a19703acc1751bbf4501fad69a6da6530090ca52ef9 Oct 30 22:31:40.122200 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:40.122159 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" event={"ID":"876dbfb5-875f-4336-8247-982a6ac58d37","Type":"ContainerStarted","Data":"aab63736d527416674e2efea4b21b2a5ca63c3652d37460f3727e17728fb5978"} Oct 30 22:31:40.122670 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:40.122212 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" event={"ID":"876dbfb5-875f-4336-8247-982a6ac58d37","Type":"ContainerStarted","Data":"59a17a6bfe87d48c37435a19703acc1751bbf4501fad69a6da6530090ca52ef9"} Oct 30 22:31:40.122670 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:40.122252 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" Oct 30 22:31:40.163759 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:40.163703 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" podStartSLOduration=2.163688838 podStartE2EDuration="2.163688838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:31:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:31:40.162591835 +0000 UTC m=+1417.078278149" watchObservedRunningTime="2026-10-30 22:31:40.163688838 +0000 UTC m=+1417.079375140" Oct 30 22:31:43.035498 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:43.035458 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" podUID="f0a58038-763f-4cd6-8921-c073bf85cb81" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused" Oct 30 22:31:46.132878 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:46.132849 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" Oct 30 22:31:50.889301 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:50.889262 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:31:53.035003 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:31:53.034957 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" podUID="f0a58038-763f-4cd6-8921-c073bf85cb81" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused" Oct 30 22:31:59.763212 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:31:59.763170 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:32:03.035375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:03.035333 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" podUID="f0a58038-763f-4cd6-8921-c073bf85cb81" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused" Oct 30 22:32:13.035246 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:13.035211 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" Oct 30 22:32:24.306385 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:24.306347 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r"] Oct 30 22:32:24.312645 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:24.312614 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" Oct 30 22:32:24.326980 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:24.326949 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:32:24.328900 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:24.328878 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r"] Oct 30 22:32:24.338925 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:24.338904 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-1b4518b221\"/\"sequence-graph-15dee-serving-cert\"" Oct 30 22:32:24.338925 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:24.338913 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1b4518b221\"/\"sequence-graph-15dee-kube-rbac-proxy-sar-config\"" Oct 30 22:32:24.366843 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:24.366810 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/490439c0-06df-43be-80ce-60586dd461c9-proxy-tls\") pod \"sequence-graph-15dee-557b5b7697-lmt2r\" (UID: \"490439c0-06df-43be-80ce-60586dd461c9\") " pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" Oct 30 22:32:24.366969 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:24.366872 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490439c0-06df-43be-80ce-60586dd461c9-openshift-service-ca-bundle\") pod \"sequence-graph-15dee-557b5b7697-lmt2r\" (UID: \"490439c0-06df-43be-80ce-60586dd461c9\") " pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" Oct 30 22:32:24.456650 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:24.456618 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:32:24.467776 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:24.467753 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/490439c0-06df-43be-80ce-60586dd461c9-proxy-tls\") pod \"sequence-graph-15dee-557b5b7697-lmt2r\" (UID: \"490439c0-06df-43be-80ce-60586dd461c9\") " pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" Oct 30 22:32:24.467850 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:24.467798 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490439c0-06df-43be-80ce-60586dd461c9-openshift-service-ca-bundle\") pod \"sequence-graph-15dee-557b5b7697-lmt2r\" (UID: \"490439c0-06df-43be-80ce-60586dd461c9\") " pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" Oct 30 22:32:24.468398 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:24.468376 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490439c0-06df-43be-80ce-60586dd461c9-openshift-service-ca-bundle\") pod \"sequence-graph-15dee-557b5b7697-lmt2r\" (UID: \"490439c0-06df-43be-80ce-60586dd461c9\") " pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" Oct 30 22:32:24.470108 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:24.470086 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/490439c0-06df-43be-80ce-60586dd461c9-proxy-tls\") pod \"sequence-graph-15dee-557b5b7697-lmt2r\" (UID: \"490439c0-06df-43be-80ce-60586dd461c9\") " pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" Oct 30 22:32:24.625175 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:24.625088 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" Oct 30 22:32:24.919739 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:24.919712 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r"] Oct 30 22:32:24.921261 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:32:24.921235 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod490439c0_06df_43be_80ce_60586dd461c9.slice/crio-85c11b1b92531e1cf54ccc6c035b19ed177ba17cd301c006db51af6087444142 WatchSource:0}: Error finding container 85c11b1b92531e1cf54ccc6c035b19ed177ba17cd301c006db51af6087444142: Status 404 returned error can't find the container with id 85c11b1b92531e1cf54ccc6c035b19ed177ba17cd301c006db51af6087444142 Oct 30 22:32:25.318869 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:25.318788 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" event={"ID":"490439c0-06df-43be-80ce-60586dd461c9","Type":"ContainerStarted","Data":"efc34ce3d208a8eb2e342483364956a1acc3483e07df0de2911ac96b8f788f88"} Oct 30 22:32:25.318869 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:25.318845 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" event={"ID":"490439c0-06df-43be-80ce-60586dd461c9","Type":"ContainerStarted","Data":"85c11b1b92531e1cf54ccc6c035b19ed177ba17cd301c006db51af6087444142"} Oct 30 22:32:25.319287 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:25.318894 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" Oct 30 22:32:25.359728 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:25.359682 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" podStartSLOduration=1.359668423 podStartE2EDuration="1.359668423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:32:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:32:25.357821968 +0000 UTC m=+1462.273508269" watchObservedRunningTime="2026-10-30 22:32:25.359668423 +0000 UTC m=+1462.275354719" Oct 30 22:32:29.763094 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:32:29.763054 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:32:31.327948 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:31.327918 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" Oct 30 22:32:51.007860 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:32:51.007816 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:32:59.763914 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:32:59.763872 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:33:03.378815 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:33:03.378786 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:33:03.396347 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:33:03.396320 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:33:29.763215 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:33:29.763174 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:33:51.143478 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:33:51.143437 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:33:52.240571 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:33:52.240535 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:33:59.763152 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:33:59.763113 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:34:18.270274 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:34:18.270190 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:34:29.762631 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:34:29.762595 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:34:51.282408 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:34:51.282367 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:34:59.763113 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:34:59.763065 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:35:29.762447 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:35:29.762405 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:35:51.350559 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:35:51.350485 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:35:52.369487 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:35:52.369461 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:35:59.763067 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:35:59.763015 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:36:29.762798 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:36:29.762763 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:36:44.904793 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:36:44.904760 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:36:51.444344 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:36:51.444309 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:36:59.763453 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:36:59.763368 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:37:23.250428 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:37:23.250392 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:37:24.972569 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:37:24.972533 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:37:29.763003 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:37:29.762970 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:37:31.179433 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:37:31.179391 4615 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" podUID="f39ce36b79a5ed24b44c91f0e6e003f7" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 22:37:51.631785 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:37:51.631741 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:37:59.763441 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:37:59.763407 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:38:03.407116 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:38:03.407077 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:38:03.420763 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:38:03.420743 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:38:29.762205 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:38:29.762168 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:38:51.725605 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:38:51.725522 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:38:59.762909 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:38:59.762874 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:39:29.762572 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:39:29.762535 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:39:31.179136 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:31.179083 4615 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-94.ec2.internal" podUID="f39ce36b79a5ed24b44c91f0e6e003f7" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 22:39:46.809275 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:46.809239 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:39:51.890059 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:51.890020 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:39:54.741381 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:54.741345 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j"] Oct 30 22:39:54.741828 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:54.741578 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" podUID="7c365bd6-8878-43b6-b942-4098b4df006b" containerName="kserve-container" containerID="cri-o://dc01a3da9b2c60992ce69cefb59a2c054d9d54640a4de817a5454e07a64fd223" gracePeriod=30 Oct 30 22:39:54.741828 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:54.741617 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" podUID="7c365bd6-8878-43b6-b942-4098b4df006b" containerName="kube-rbac-proxy" containerID="cri-o://3ed1b7767bfa8e35cc397fe68a61046900fdf9905997eb8585ed47f359db0299" gracePeriod=30 Oct 30 22:39:54.810418 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:54.810388 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:39:54.936209 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:54.936179 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:39:55.196768 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:55.196734 4615 generic.go:356] "Generic (PLEG): container finished" podID="7c365bd6-8878-43b6-b942-4098b4df006b" containerID="3ed1b7767bfa8e35cc397fe68a61046900fdf9905997eb8585ed47f359db0299" exitCode=0 Oct 30 22:39:55.196944 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:55.196805 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" event={"ID":"7c365bd6-8878-43b6-b942-4098b4df006b","Type":"ContainerDied","Data":"3ed1b7767bfa8e35cc397fe68a61046900fdf9905997eb8585ed47f359db0299"} Oct 30 22:39:57.832532 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:57.832485 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" podUID="7c365bd6-8878-43b6-b942-4098b4df006b" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.36:8643/healthz\": dial tcp 10.134.0.36:8643: connect: connection refused" Oct 30 22:39:57.836884 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:57.836851 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" podUID="7c365bd6-8878-43b6-b942-4098b4df006b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused" Oct 30 22:39:58.385945 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:58.385916 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" Oct 30 22:39:58.427739 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:58.427706 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7c365bd6-8878-43b6-b942-4098b4df006b-proxy-tls\" (UniqueName: \"kubernetes.io/secret/7c365bd6-8878-43b6-b942-4098b4df006b-proxy-tls\") pod \"7c365bd6-8878-43b6-b942-4098b4df006b\" (UID: \"7c365bd6-8878-43b6-b942-4098b4df006b\") " Oct 30 22:39:58.427979 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:58.427825 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c365bd6-8878-43b6-b942-4098b4df006b-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7c365bd6-8878-43b6-b942-4098b4df006b-kube-rbac-proxy-sar-config\") pod \"7c365bd6-8878-43b6-b942-4098b4df006b\" (UID: \"7c365bd6-8878-43b6-b942-4098b4df006b\") " Oct 30 22:39:58.427979 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:58.427865 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c365bd6-8878-43b6-b942-4098b4df006b-kube-api-access-vq74j\" (UniqueName: \"kubernetes.io/projected/7c365bd6-8878-43b6-b942-4098b4df006b-kube-api-access-vq74j\") pod \"7c365bd6-8878-43b6-b942-4098b4df006b\" (UID: \"7c365bd6-8878-43b6-b942-4098b4df006b\") " Oct 30 22:39:58.428263 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:58.428234 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c365bd6-8878-43b6-b942-4098b4df006b-kube-rbac-proxy-sar-config" pod "7c365bd6-8878-43b6-b942-4098b4df006b" (UID: "7c365bd6-8878-43b6-b942-4098b4df006b"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:39:58.429980 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:58.429950 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c365bd6-8878-43b6-b942-4098b4df006b-kube-api-access-vq74j" pod "7c365bd6-8878-43b6-b942-4098b4df006b" (UID: "7c365bd6-8878-43b6-b942-4098b4df006b"). InnerVolumeSpecName "kube-api-access-vq74j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:39:58.430251 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:58.430204 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c365bd6-8878-43b6-b942-4098b4df006b-proxy-tls" pod "7c365bd6-8878-43b6-b942-4098b4df006b" (UID: "7c365bd6-8878-43b6-b942-4098b4df006b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:39:58.529305 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:58.529268 4615 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7c365bd6-8878-43b6-b942-4098b4df006b-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:39:58.529305 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:58.529298 4615 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vq74j\" (UniqueName: \"kubernetes.io/projected/7c365bd6-8878-43b6-b942-4098b4df006b-kube-api-access-vq74j\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:39:58.529305 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:58.529310 4615 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7c365bd6-8878-43b6-b942-4098b4df006b-proxy-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:39:59.214354 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:59.214316 4615 generic.go:356] "Generic (PLEG): container finished" podID="7c365bd6-8878-43b6-b942-4098b4df006b" containerID="dc01a3da9b2c60992ce69cefb59a2c054d9d54640a4de817a5454e07a64fd223" exitCode=0 Oct 30 22:39:59.214792 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:59.214373 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" Oct 30 22:39:59.214792 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:59.214401 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" event={"ID":"7c365bd6-8878-43b6-b942-4098b4df006b","Type":"ContainerDied","Data":"dc01a3da9b2c60992ce69cefb59a2c054d9d54640a4de817a5454e07a64fd223"} Oct 30 22:39:59.214792 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:59.214461 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j" event={"ID":"7c365bd6-8878-43b6-b942-4098b4df006b","Type":"ContainerDied","Data":"848af9c804a908353fce8c1a97c2f4e4ce835c39e1c644473e5f313d3ae5a6d8"} Oct 30 22:39:59.214792 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:59.214487 4615 scope.go:122] "RemoveContainer" containerID="3ed1b7767bfa8e35cc397fe68a61046900fdf9905997eb8585ed47f359db0299" Oct 30 22:39:59.222941 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:59.222921 4615 scope.go:122] "RemoveContainer" containerID="dc01a3da9b2c60992ce69cefb59a2c054d9d54640a4de817a5454e07a64fd223" Oct 30 22:39:59.230516 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:59.230501 4615 scope.go:122] "RemoveContainer" containerID="3ed1b7767bfa8e35cc397fe68a61046900fdf9905997eb8585ed47f359db0299" Oct 30 22:39:59.230843 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:39:59.230822 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ed1b7767bfa8e35cc397fe68a61046900fdf9905997eb8585ed47f359db0299\": container with ID starting with 3ed1b7767bfa8e35cc397fe68a61046900fdf9905997eb8585ed47f359db0299 not found: ID does not exist" containerID="3ed1b7767bfa8e35cc397fe68a61046900fdf9905997eb8585ed47f359db0299" Oct 30 22:39:59.230894 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:59.230853 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ed1b7767bfa8e35cc397fe68a61046900fdf9905997eb8585ed47f359db0299"} err="failed to get container status \"3ed1b7767bfa8e35cc397fe68a61046900fdf9905997eb8585ed47f359db0299\": rpc error: code = NotFound desc = could not find container \"3ed1b7767bfa8e35cc397fe68a61046900fdf9905997eb8585ed47f359db0299\": container with ID starting with 3ed1b7767bfa8e35cc397fe68a61046900fdf9905997eb8585ed47f359db0299 not found: ID does not exist" Oct 30 22:39:59.230894 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:59.230870 4615 scope.go:122] "RemoveContainer" containerID="dc01a3da9b2c60992ce69cefb59a2c054d9d54640a4de817a5454e07a64fd223" Oct 30 22:39:59.231194 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:39:59.231157 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc01a3da9b2c60992ce69cefb59a2c054d9d54640a4de817a5454e07a64fd223\": container with ID starting with dc01a3da9b2c60992ce69cefb59a2c054d9d54640a4de817a5454e07a64fd223 not found: ID does not exist" containerID="dc01a3da9b2c60992ce69cefb59a2c054d9d54640a4de817a5454e07a64fd223" Oct 30 22:39:59.231288 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:59.231205 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc01a3da9b2c60992ce69cefb59a2c054d9d54640a4de817a5454e07a64fd223"} err="failed to get container status \"dc01a3da9b2c60992ce69cefb59a2c054d9d54640a4de817a5454e07a64fd223\": rpc error: code = NotFound desc = could not find container \"dc01a3da9b2c60992ce69cefb59a2c054d9d54640a4de817a5454e07a64fd223\": container with ID starting with dc01a3da9b2c60992ce69cefb59a2c054d9d54640a4de817a5454e07a64fd223 not found: ID does not exist" Oct 30 22:39:59.264452 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:59.264416 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j"] Oct 30 22:39:59.276172 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:39:59.276146 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-83e67-predictor-767594485d-svp5j"] Oct 30 22:39:59.762674 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:39:59.762640 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:40:00.073280 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.073198 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:40:00.189312 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.189275 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:40:00.758237 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.758206 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:40:00.798135 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.798102 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc"] Oct 30 22:40:00.798444 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.798432 4615 cpu_manager.go:414] "containerMap: removing container" podUID="7c365bd6-8878-43b6-b942-4098b4df006b" containerName="kserve-container" Oct 30 22:40:00.798501 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.798445 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c365bd6-8878-43b6-b942-4098b4df006b" containerName="kserve-container" Oct 30 22:40:00.798501 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.798460 4615 cpu_manager.go:414] "containerMap: removing container" podUID="7c365bd6-8878-43b6-b942-4098b4df006b" containerName="kube-rbac-proxy" Oct 30 22:40:00.798501 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.798465 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c365bd6-8878-43b6-b942-4098b4df006b" containerName="kube-rbac-proxy" Oct 30 22:40:00.798620 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.798528 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c365bd6-8878-43b6-b942-4098b4df006b" containerName="kube-rbac-proxy" Oct 30 22:40:00.798620 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.798541 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c365bd6-8878-43b6-b942-4098b4df006b" containerName="kserve-container" Oct 30 22:40:00.803717 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.803698 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" Oct 30 22:40:00.820771 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.820744 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc"] Oct 30 22:40:00.831280 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.831261 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-cc765036c8\"/\"kube-root-ca.crt\"" Oct 30 22:40:00.831280 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.831261 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-cc765036c8\"/\"error-404-isvc-c00ac-kube-rbac-proxy-sar-config\"" Oct 30 22:40:00.831581 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.831292 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-cc765036c8\"/\"error-404-isvc-c00ac-predictor-serving-cert\"" Oct 30 22:40:00.849632 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.849609 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d59524f5-bf80-4785-8038-5a93d6fead9b-proxy-tls\") pod \"error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc\" (UID: \"d59524f5-bf80-4785-8038-5a93d6fead9b\") " pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" Oct 30 22:40:00.849725 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.849658 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmx89\" (UniqueName: \"kubernetes.io/projected/d59524f5-bf80-4785-8038-5a93d6fead9b-kube-api-access-zmx89\") pod \"error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc\" (UID: \"d59524f5-bf80-4785-8038-5a93d6fead9b\") " pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" Oct 30 22:40:00.849725 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.849678 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d59524f5-bf80-4785-8038-5a93d6fead9b-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc\" (UID: \"d59524f5-bf80-4785-8038-5a93d6fead9b\") " pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" Oct 30 22:40:00.877656 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.877630 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:40:00.950296 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.950267 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmx89\" (UniqueName: \"kubernetes.io/projected/d59524f5-bf80-4785-8038-5a93d6fead9b-kube-api-access-zmx89\") pod \"error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc\" (UID: \"d59524f5-bf80-4785-8038-5a93d6fead9b\") " pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" Oct 30 22:40:00.950424 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.950304 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d59524f5-bf80-4785-8038-5a93d6fead9b-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc\" (UID: \"d59524f5-bf80-4785-8038-5a93d6fead9b\") " pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" Oct 30 22:40:00.950424 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.950371 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d59524f5-bf80-4785-8038-5a93d6fead9b-proxy-tls\") pod \"error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc\" (UID: \"d59524f5-bf80-4785-8038-5a93d6fead9b\") " pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" Oct 30 22:40:00.951010 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.950985 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d59524f5-bf80-4785-8038-5a93d6fead9b-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc\" (UID: \"d59524f5-bf80-4785-8038-5a93d6fead9b\") " pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" Oct 30 22:40:00.952677 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.952660 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d59524f5-bf80-4785-8038-5a93d6fead9b-proxy-tls\") pod \"error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc\" (UID: \"d59524f5-bf80-4785-8038-5a93d6fead9b\") " pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" Oct 30 22:40:00.968744 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:00.968716 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmx89\" (UniqueName: \"kubernetes.io/projected/d59524f5-bf80-4785-8038-5a93d6fead9b-kube-api-access-zmx89\") pod \"error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc\" (UID: \"d59524f5-bf80-4785-8038-5a93d6fead9b\") " pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" Oct 30 22:40:01.118621 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:01.118531 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" Oct 30 22:40:01.233292 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:01.233259 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7c365bd6-8878-43b6-b942-4098b4df006b" path="/var/lib/kubelet/pods/7c365bd6-8878-43b6-b942-4098b4df006b/volumes" Oct 30 22:40:01.411142 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:01.411115 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc"] Oct 30 22:40:01.412668 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:40:01.412639 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd59524f5_bf80_4785_8038_5a93d6fead9b.slice/crio-527ab774c39d2d8f19c2898dca65f51755bf1ed8b217f755864e2a32b1fba8f2 WatchSource:0}: Error finding container 527ab774c39d2d8f19c2898dca65f51755bf1ed8b217f755864e2a32b1fba8f2: Status 404 returned error can't find the container with id 527ab774c39d2d8f19c2898dca65f51755bf1ed8b217f755864e2a32b1fba8f2 Oct 30 22:40:02.229969 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:02.229935 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" event={"ID":"d59524f5-bf80-4785-8038-5a93d6fead9b","Type":"ContainerStarted","Data":"20e5f9d8e3c95a7d00c3230bd70a692c4163e81202fff33a8c13662d819cac89"} Oct 30 22:40:02.230431 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:02.229998 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" event={"ID":"d59524f5-bf80-4785-8038-5a93d6fead9b","Type":"ContainerStarted","Data":"e7115c90845d4d1c482af2d4a436b6ca99857a4baa16ed6e91457ca4314e1dab"} Oct 30 22:40:02.230431 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:02.230025 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" Oct 30 22:40:02.230431 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:02.230061 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" Oct 30 22:40:02.230431 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:02.230086 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" event={"ID":"d59524f5-bf80-4785-8038-5a93d6fead9b","Type":"ContainerStarted","Data":"527ab774c39d2d8f19c2898dca65f51755bf1ed8b217f755864e2a32b1fba8f2"} Oct 30 22:40:02.231457 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:02.231432 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" podUID="d59524f5-bf80-4785-8038-5a93d6fead9b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused" Oct 30 22:40:02.268244 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:02.268189 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" podStartSLOduration=2.268176061 podStartE2EDuration="2.268176061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:40:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:40:02.268017081 +0000 UTC m=+1919.183703405" watchObservedRunningTime="2026-10-30 22:40:02.268176061 +0000 UTC m=+1919.183862363" Oct 30 22:40:03.234302 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:03.234261 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" podUID="d59524f5-bf80-4785-8038-5a93d6fead9b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused" Oct 30 22:40:08.238735 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:08.238708 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" Oct 30 22:40:08.239225 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:08.239196 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" podUID="d59524f5-bf80-4785-8038-5a93d6fead9b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused" Oct 30 22:40:18.239787 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:18.239693 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" podUID="d59524f5-bf80-4785-8038-5a93d6fead9b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused" Oct 30 22:40:28.239822 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:28.239788 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" podUID="d59524f5-bf80-4785-8038-5a93d6fead9b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused" Oct 30 22:40:29.762840 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:40:29.762805 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:40:38.239978 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:38.239939 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" podUID="d59524f5-bf80-4785-8038-5a93d6fead9b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused" Oct 30 22:40:39.780237 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:39.780204 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5"] Oct 30 22:40:39.780630 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:39.780430 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" podUID="f0a58038-763f-4cd6-8921-c073bf85cb81" containerName="kserve-container" containerID="cri-o://1621826ec6011fbb862e07be7ae7c672b82a7e8e135c42f8c9140030caeef230" gracePeriod=30 Oct 30 22:40:39.780630 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:39.780458 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" podUID="f0a58038-763f-4cd6-8921-c073bf85cb81" containerName="kube-rbac-proxy" containerID="cri-o://9e62abcaaa0eb1782b12dcc94440f9690366a29328b74becd540e0c22e1e0c17" gracePeriod=30 Oct 30 22:40:39.843181 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:39.843158 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:40:39.961333 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:39.961298 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:40:40.381420 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:40.381385 4615 generic.go:356] "Generic (PLEG): container finished" podID="f0a58038-763f-4cd6-8921-c073bf85cb81" containerID="9e62abcaaa0eb1782b12dcc94440f9690366a29328b74becd540e0c22e1e0c17" exitCode=0 Oct 30 22:40:40.381623 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:40.381464 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" event={"ID":"f0a58038-763f-4cd6-8921-c073bf85cb81","Type":"ContainerDied","Data":"9e62abcaaa0eb1782b12dcc94440f9690366a29328b74becd540e0c22e1e0c17"} Oct 30 22:40:43.030410 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:43.030365 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" podUID="f0a58038-763f-4cd6-8921-c073bf85cb81" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.38:8643/healthz\": dial tcp 10.134.0.38:8643: connect: connection refused" Oct 30 22:40:43.034670 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:43.034649 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" podUID="f0a58038-763f-4cd6-8921-c073bf85cb81" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused" Oct 30 22:40:43.396053 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:43.396010 4615 generic.go:356] "Generic (PLEG): container finished" podID="f0a58038-763f-4cd6-8921-c073bf85cb81" containerID="1621826ec6011fbb862e07be7ae7c672b82a7e8e135c42f8c9140030caeef230" exitCode=0 Oct 30 22:40:43.396169 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:43.396074 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" event={"ID":"f0a58038-763f-4cd6-8921-c073bf85cb81","Type":"ContainerDied","Data":"1621826ec6011fbb862e07be7ae7c672b82a7e8e135c42f8c9140030caeef230"} Oct 30 22:40:43.411773 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:43.411754 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" Oct 30 22:40:43.500241 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:43.500214 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0a58038-763f-4cd6-8921-c073bf85cb81-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0a58038-763f-4cd6-8921-c073bf85cb81-kube-rbac-proxy-sar-config\") pod \"f0a58038-763f-4cd6-8921-c073bf85cb81\" (UID: \"f0a58038-763f-4cd6-8921-c073bf85cb81\") " Oct 30 22:40:43.500435 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:43.500289 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f0a58038-763f-4cd6-8921-c073bf85cb81-proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0a58038-763f-4cd6-8921-c073bf85cb81-proxy-tls\") pod \"f0a58038-763f-4cd6-8921-c073bf85cb81\" (UID: \"f0a58038-763f-4cd6-8921-c073bf85cb81\") " Oct 30 22:40:43.500435 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:43.500375 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0a58038-763f-4cd6-8921-c073bf85cb81-kube-api-access-jrnzt\" (UniqueName: \"kubernetes.io/projected/f0a58038-763f-4cd6-8921-c073bf85cb81-kube-api-access-jrnzt\") pod \"f0a58038-763f-4cd6-8921-c073bf85cb81\" (UID: \"f0a58038-763f-4cd6-8921-c073bf85cb81\") " Oct 30 22:40:43.500656 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:43.500628 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0a58038-763f-4cd6-8921-c073bf85cb81-kube-rbac-proxy-sar-config" pod "f0a58038-763f-4cd6-8921-c073bf85cb81" (UID: "f0a58038-763f-4cd6-8921-c073bf85cb81"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:40:43.502382 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:43.502359 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0a58038-763f-4cd6-8921-c073bf85cb81-kube-api-access-jrnzt" pod "f0a58038-763f-4cd6-8921-c073bf85cb81" (UID: "f0a58038-763f-4cd6-8921-c073bf85cb81"). InnerVolumeSpecName "kube-api-access-jrnzt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:40:43.502487 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:43.502385 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0a58038-763f-4cd6-8921-c073bf85cb81-proxy-tls" pod "f0a58038-763f-4cd6-8921-c073bf85cb81" (UID: "f0a58038-763f-4cd6-8921-c073bf85cb81"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:40:43.601161 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:43.601080 4615 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f0a58038-763f-4cd6-8921-c073bf85cb81-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:40:43.601161 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:43.601109 4615 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f0a58038-763f-4cd6-8921-c073bf85cb81-proxy-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:40:43.601161 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:43.601121 4615 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrnzt\" (UniqueName: \"kubernetes.io/projected/f0a58038-763f-4cd6-8921-c073bf85cb81-kube-api-access-jrnzt\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:40:44.402248 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:44.402218 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" Oct 30 22:40:44.402678 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:44.402218 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5" event={"ID":"f0a58038-763f-4cd6-8921-c073bf85cb81","Type":"ContainerDied","Data":"11db88d1eb5e402092d98157ef4bdbe43cadbf27615158e040101a5c40e022ea"} Oct 30 22:40:44.402678 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:44.402358 4615 scope.go:122] "RemoveContainer" containerID="9e62abcaaa0eb1782b12dcc94440f9690366a29328b74becd540e0c22e1e0c17" Oct 30 22:40:44.410419 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:44.410404 4615 scope.go:122] "RemoveContainer" containerID="1621826ec6011fbb862e07be7ae7c672b82a7e8e135c42f8c9140030caeef230" Oct 30 22:40:44.452243 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:44.452219 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5"] Oct 30 22:40:44.464039 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:44.464020 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-15dee-predictor-66f695dc6c-fcvf5"] Oct 30 22:40:45.226772 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:45.226731 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:40:45.234025 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:45.233997 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0a58038-763f-4cd6-8921-c073bf85cb81" path="/var/lib/kubelet/pods/f0a58038-763f-4cd6-8921-c073bf85cb81/volumes" Oct 30 22:40:45.345126 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:45.345097 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:40:45.893277 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:45.893245 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:40:45.932133 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:45.932104 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt"] Oct 30 22:40:45.932445 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:45.932434 4615 cpu_manager.go:414] "containerMap: removing container" podUID="f0a58038-763f-4cd6-8921-c073bf85cb81" containerName="kube-rbac-proxy" Oct 30 22:40:45.932486 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:45.932447 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0a58038-763f-4cd6-8921-c073bf85cb81" containerName="kube-rbac-proxy" Oct 30 22:40:45.932486 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:45.932475 4615 cpu_manager.go:414] "containerMap: removing container" podUID="f0a58038-763f-4cd6-8921-c073bf85cb81" containerName="kserve-container" Oct 30 22:40:45.932486 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:45.932480 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0a58038-763f-4cd6-8921-c073bf85cb81" containerName="kserve-container" Oct 30 22:40:45.932581 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:45.932530 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0a58038-763f-4cd6-8921-c073bf85cb81" containerName="kube-rbac-proxy" Oct 30 22:40:45.932581 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:45.932542 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0a58038-763f-4cd6-8921-c073bf85cb81" containerName="kserve-container" Oct 30 22:40:45.937105 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:45.937091 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" Oct 30 22:40:45.954976 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:45.954952 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt"] Oct 30 22:40:45.965328 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:45.965307 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1b4518b221\"/\"error-404-isvc-504c0-kube-rbac-proxy-sar-config\"" Oct 30 22:40:45.965467 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:45.965347 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-1b4518b221\"/\"error-404-isvc-504c0-predictor-serving-cert\"" Oct 30 22:40:45.965467 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:45.965369 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1b4518b221\"/\"kube-root-ca.crt\"" Oct 30 22:40:46.014869 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:46.014839 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:40:46.020927 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:46.020907 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9b7c3e3f-a09b-4eaf-91a9-f7809e028efd-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-504c0-predictor-f6b97589f-72rzt\" (UID: \"9b7c3e3f-a09b-4eaf-91a9-f7809e028efd\") " pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" Oct 30 22:40:46.021007 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:46.020957 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9wq9\" (UniqueName: \"kubernetes.io/projected/9b7c3e3f-a09b-4eaf-91a9-f7809e028efd-kube-api-access-h9wq9\") pod \"error-404-isvc-504c0-predictor-f6b97589f-72rzt\" (UID: \"9b7c3e3f-a09b-4eaf-91a9-f7809e028efd\") " pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" Oct 30 22:40:46.021086 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:46.021060 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9b7c3e3f-a09b-4eaf-91a9-f7809e028efd-proxy-tls\") pod \"error-404-isvc-504c0-predictor-f6b97589f-72rzt\" (UID: \"9b7c3e3f-a09b-4eaf-91a9-f7809e028efd\") " pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" Oct 30 22:40:46.122418 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:46.122380 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9b7c3e3f-a09b-4eaf-91a9-f7809e028efd-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-504c0-predictor-f6b97589f-72rzt\" (UID: \"9b7c3e3f-a09b-4eaf-91a9-f7809e028efd\") " pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" Oct 30 22:40:46.122607 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:46.122464 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9wq9\" (UniqueName: \"kubernetes.io/projected/9b7c3e3f-a09b-4eaf-91a9-f7809e028efd-kube-api-access-h9wq9\") pod \"error-404-isvc-504c0-predictor-f6b97589f-72rzt\" (UID: \"9b7c3e3f-a09b-4eaf-91a9-f7809e028efd\") " pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" Oct 30 22:40:46.122607 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:46.122503 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9b7c3e3f-a09b-4eaf-91a9-f7809e028efd-proxy-tls\") pod \"error-404-isvc-504c0-predictor-f6b97589f-72rzt\" (UID: \"9b7c3e3f-a09b-4eaf-91a9-f7809e028efd\") " pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" Oct 30 22:40:46.123034 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:46.123009 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9b7c3e3f-a09b-4eaf-91a9-f7809e028efd-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-504c0-predictor-f6b97589f-72rzt\" (UID: \"9b7c3e3f-a09b-4eaf-91a9-f7809e028efd\") " pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" Oct 30 22:40:46.124737 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:46.124719 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9b7c3e3f-a09b-4eaf-91a9-f7809e028efd-proxy-tls\") pod \"error-404-isvc-504c0-predictor-f6b97589f-72rzt\" (UID: \"9b7c3e3f-a09b-4eaf-91a9-f7809e028efd\") " pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" Oct 30 22:40:46.140972 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:46.140943 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9wq9\" (UniqueName: \"kubernetes.io/projected/9b7c3e3f-a09b-4eaf-91a9-f7809e028efd-kube-api-access-h9wq9\") pod \"error-404-isvc-504c0-predictor-f6b97589f-72rzt\" (UID: \"9b7c3e3f-a09b-4eaf-91a9-f7809e028efd\") " pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" Oct 30 22:40:46.251714 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:46.251632 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" Oct 30 22:40:46.541937 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:46.541910 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt"] Oct 30 22:40:46.542461 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:40:46.542437 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b7c3e3f_a09b_4eaf_91a9_f7809e028efd.slice/crio-26ffbc52d9af20f2ec66eaa4977ae2295eba38202b77e6634ef7fab750bc4a2f WatchSource:0}: Error finding container 26ffbc52d9af20f2ec66eaa4977ae2295eba38202b77e6634ef7fab750bc4a2f: Status 404 returned error can't find the container with id 26ffbc52d9af20f2ec66eaa4977ae2295eba38202b77e6634ef7fab750bc4a2f Oct 30 22:40:47.417716 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:47.417678 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" event={"ID":"9b7c3e3f-a09b-4eaf-91a9-f7809e028efd","Type":"ContainerStarted","Data":"188058239a776b6f3078a2fe3f631393d0d131b28a4325d7ceb89eb7b8a971ff"} Oct 30 22:40:47.418147 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:47.417748 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" event={"ID":"9b7c3e3f-a09b-4eaf-91a9-f7809e028efd","Type":"ContainerStarted","Data":"888cd75691c5d9471fb3fac3edd126e568ac4590b9c3d96740b5c29e7ce65be9"} Oct 30 22:40:47.418147 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:47.417782 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" Oct 30 22:40:47.418147 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:47.417808 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" event={"ID":"9b7c3e3f-a09b-4eaf-91a9-f7809e028efd","Type":"ContainerStarted","Data":"26ffbc52d9af20f2ec66eaa4977ae2295eba38202b77e6634ef7fab750bc4a2f"} Oct 30 22:40:47.456014 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:47.455952 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" podStartSLOduration=2.45593761 podStartE2EDuration="2.45593761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:40:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:40:47.455574274 +0000 UTC m=+1964.371260576" watchObservedRunningTime="2026-10-30 22:40:47.45593761 +0000 UTC m=+1964.371623912" Oct 30 22:40:48.239110 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:48.239074 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" podUID="d59524f5-bf80-4785-8038-5a93d6fead9b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused" Oct 30 22:40:48.427342 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:48.427309 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" Oct 30 22:40:48.428716 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:48.428689 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" podUID="9b7c3e3f-a09b-4eaf-91a9-f7809e028efd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused" Oct 30 22:40:49.430836 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:49.430792 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" podUID="9b7c3e3f-a09b-4eaf-91a9-f7809e028efd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused" Oct 30 22:40:51.908608 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:51.908576 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:40:52.404296 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:52.404261 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:40:53.057698 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:53.057663 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:40:54.435121 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:54.435095 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" Oct 30 22:40:54.435559 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:54.435534 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" podUID="9b7c3e3f-a09b-4eaf-91a9-f7809e028efd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused" Oct 30 22:40:58.240108 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:40:58.240078 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" Oct 30 22:40:59.763074 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:40:59.763024 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:41:04.435889 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:04.435843 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" podUID="9b7c3e3f-a09b-4eaf-91a9-f7809e028efd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused" Oct 30 22:41:10.318158 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:10.318124 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs"] Oct 30 22:41:10.322817 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:10.322799 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" Oct 30 22:41:10.338233 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:10.338209 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-cc765036c8\"/\"splitter-graph-c00ac-serving-cert\"" Oct 30 22:41:10.338630 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:10.338617 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-cc765036c8\"/\"splitter-graph-c00ac-kube-rbac-proxy-sar-config\"" Oct 30 22:41:10.351718 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:10.351692 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs"] Oct 30 22:41:10.368905 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:10.368879 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:41:10.415379 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:10.415352 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a58e4278-3be4-415f-90f2-6a3c42afbb6f-openshift-service-ca-bundle\") pod \"splitter-graph-c00ac-fdb955b67-67rgs\" (UID: \"a58e4278-3be4-415f-90f2-6a3c42afbb6f\") " pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" Oct 30 22:41:10.415499 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:10.415427 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a58e4278-3be4-415f-90f2-6a3c42afbb6f-proxy-tls\") pod \"splitter-graph-c00ac-fdb955b67-67rgs\" (UID: \"a58e4278-3be4-415f-90f2-6a3c42afbb6f\") " pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" Oct 30 22:41:10.516380 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:10.516345 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a58e4278-3be4-415f-90f2-6a3c42afbb6f-openshift-service-ca-bundle\") pod \"splitter-graph-c00ac-fdb955b67-67rgs\" (UID: \"a58e4278-3be4-415f-90f2-6a3c42afbb6f\") " pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" Oct 30 22:41:10.516561 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:10.516446 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a58e4278-3be4-415f-90f2-6a3c42afbb6f-proxy-tls\") pod \"splitter-graph-c00ac-fdb955b67-67rgs\" (UID: \"a58e4278-3be4-415f-90f2-6a3c42afbb6f\") " pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" Oct 30 22:41:10.516963 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:10.516940 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a58e4278-3be4-415f-90f2-6a3c42afbb6f-openshift-service-ca-bundle\") pod \"splitter-graph-c00ac-fdb955b67-67rgs\" (UID: \"a58e4278-3be4-415f-90f2-6a3c42afbb6f\") " pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" Oct 30 22:41:10.518829 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:10.518806 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a58e4278-3be4-415f-90f2-6a3c42afbb6f-proxy-tls\") pod \"splitter-graph-c00ac-fdb955b67-67rgs\" (UID: \"a58e4278-3be4-415f-90f2-6a3c42afbb6f\") " pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" Oct 30 22:41:10.546507 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:10.546471 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:41:10.635052 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:10.635025 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" Oct 30 22:41:10.924442 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:10.924415 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs"] Oct 30 22:41:10.925386 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:41:10.925359 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda58e4278_3be4_415f_90f2_6a3c42afbb6f.slice/crio-d6866bbe540d907c95198da53b1bfe70ee8d9b322eff414c4102e4ce7cade836 WatchSource:0}: Error finding container d6866bbe540d907c95198da53b1bfe70ee8d9b322eff414c4102e4ce7cade836: Status 404 returned error can't find the container with id d6866bbe540d907c95198da53b1bfe70ee8d9b322eff414c4102e4ce7cade836 Oct 30 22:41:11.518848 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:11.518815 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" event={"ID":"a58e4278-3be4-415f-90f2-6a3c42afbb6f","Type":"ContainerStarted","Data":"d5734dbfea3666605de0e02a2f85b4a485b780e5a633f1b6c71600293325e180"} Oct 30 22:41:11.519247 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:11.518863 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" event={"ID":"a58e4278-3be4-415f-90f2-6a3c42afbb6f","Type":"ContainerStarted","Data":"d6866bbe540d907c95198da53b1bfe70ee8d9b322eff414c4102e4ce7cade836"} Oct 30 22:41:11.519247 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:11.518923 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" Oct 30 22:41:11.558059 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:11.558003 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" podStartSLOduration=1.557990708 podStartE2EDuration="1.557990708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:41:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:41:11.55667931 +0000 UTC m=+1988.472365611" watchObservedRunningTime="2026-10-30 22:41:11.557990708 +0000 UTC m=+1988.473677020" Oct 30 22:41:14.436329 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:14.436292 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" podUID="9b7c3e3f-a09b-4eaf-91a9-f7809e028efd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused" Oct 30 22:41:17.529785 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:17.529750 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" Oct 30 22:41:21.728240 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:21.728205 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc"] Oct 30 22:41:21.729120 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:21.729069 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" podUID="d59524f5-bf80-4785-8038-5a93d6fead9b" containerName="kserve-container" containerID="cri-o://e7115c90845d4d1c482af2d4a436b6ca99857a4baa16ed6e91457ca4314e1dab" gracePeriod=30 Oct 30 22:41:21.729540 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:21.729506 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" podUID="d59524f5-bf80-4785-8038-5a93d6fead9b" containerName="kube-rbac-proxy" containerID="cri-o://20e5f9d8e3c95a7d00c3230bd70a692c4163e81202fff33a8c13662d819cac89" gracePeriod=30 Oct 30 22:41:21.819280 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:21.819246 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:41:22.095341 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:22.095236 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:41:22.566717 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:22.566685 4615 generic.go:356] "Generic (PLEG): container finished" podID="d59524f5-bf80-4785-8038-5a93d6fead9b" containerID="20e5f9d8e3c95a7d00c3230bd70a692c4163e81202fff33a8c13662d819cac89" exitCode=0 Oct 30 22:41:22.566892 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:22.566723 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" event={"ID":"d59524f5-bf80-4785-8038-5a93d6fead9b","Type":"ContainerDied","Data":"20e5f9d8e3c95a7d00c3230bd70a692c4163e81202fff33a8c13662d819cac89"} Oct 30 22:41:23.161199 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:23.161156 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:41:23.234903 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:23.234867 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" podUID="d59524f5-bf80-4785-8038-5a93d6fead9b" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.41:8643/healthz\": dial tcp 10.134.0.41:8643: connect: connection refused" Oct 30 22:41:23.908537 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:23.908483 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:41:24.435526 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:24.435491 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" podUID="9b7c3e3f-a09b-4eaf-91a9-f7809e028efd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused" Oct 30 22:41:25.174086 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:25.174064 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" Oct 30 22:41:25.337761 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:25.337695 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d59524f5-bf80-4785-8038-5a93d6fead9b-proxy-tls\" (UniqueName: \"kubernetes.io/secret/d59524f5-bf80-4785-8038-5a93d6fead9b-proxy-tls\") pod \"d59524f5-bf80-4785-8038-5a93d6fead9b\" (UID: \"d59524f5-bf80-4785-8038-5a93d6fead9b\") " Oct 30 22:41:25.337907 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:25.337791 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d59524f5-bf80-4785-8038-5a93d6fead9b-kube-api-access-zmx89\" (UniqueName: \"kubernetes.io/projected/d59524f5-bf80-4785-8038-5a93d6fead9b-kube-api-access-zmx89\") pod \"d59524f5-bf80-4785-8038-5a93d6fead9b\" (UID: \"d59524f5-bf80-4785-8038-5a93d6fead9b\") " Oct 30 22:41:25.337907 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:25.337840 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d59524f5-bf80-4785-8038-5a93d6fead9b-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d59524f5-bf80-4785-8038-5a93d6fead9b-kube-rbac-proxy-sar-config\") pod \"d59524f5-bf80-4785-8038-5a93d6fead9b\" (UID: \"d59524f5-bf80-4785-8038-5a93d6fead9b\") " Oct 30 22:41:25.338282 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:25.338243 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d59524f5-bf80-4785-8038-5a93d6fead9b-kube-rbac-proxy-sar-config" pod "d59524f5-bf80-4785-8038-5a93d6fead9b" (UID: "d59524f5-bf80-4785-8038-5a93d6fead9b"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:41:25.338434 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:25.338373 4615 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d59524f5-bf80-4785-8038-5a93d6fead9b-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:41:25.340004 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:25.339980 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d59524f5-bf80-4785-8038-5a93d6fead9b-kube-api-access-zmx89" pod "d59524f5-bf80-4785-8038-5a93d6fead9b" (UID: "d59524f5-bf80-4785-8038-5a93d6fead9b"). InnerVolumeSpecName "kube-api-access-zmx89". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:41:25.340252 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:25.340227 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d59524f5-bf80-4785-8038-5a93d6fead9b-proxy-tls" pod "d59524f5-bf80-4785-8038-5a93d6fead9b" (UID: "d59524f5-bf80-4785-8038-5a93d6fead9b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:41:25.439324 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:25.439293 4615 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmx89\" (UniqueName: \"kubernetes.io/projected/d59524f5-bf80-4785-8038-5a93d6fead9b-kube-api-access-zmx89\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:41:25.439324 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:25.439319 4615 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d59524f5-bf80-4785-8038-5a93d6fead9b-proxy-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:41:25.580168 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:25.580134 4615 generic.go:356] "Generic (PLEG): container finished" podID="d59524f5-bf80-4785-8038-5a93d6fead9b" containerID="e7115c90845d4d1c482af2d4a436b6ca99857a4baa16ed6e91457ca4314e1dab" exitCode=0 Oct 30 22:41:25.580363 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:25.580189 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" Oct 30 22:41:25.580363 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:25.580190 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" event={"ID":"d59524f5-bf80-4785-8038-5a93d6fead9b","Type":"ContainerDied","Data":"e7115c90845d4d1c482af2d4a436b6ca99857a4baa16ed6e91457ca4314e1dab"} Oct 30 22:41:25.580363 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:25.580323 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc" event={"ID":"d59524f5-bf80-4785-8038-5a93d6fead9b","Type":"ContainerDied","Data":"527ab774c39d2d8f19c2898dca65f51755bf1ed8b217f755864e2a32b1fba8f2"} Oct 30 22:41:25.580363 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:25.580357 4615 scope.go:122] "RemoveContainer" containerID="20e5f9d8e3c95a7d00c3230bd70a692c4163e81202fff33a8c13662d819cac89" Oct 30 22:41:25.591470 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:25.591452 4615 scope.go:122] "RemoveContainer" containerID="e7115c90845d4d1c482af2d4a436b6ca99857a4baa16ed6e91457ca4314e1dab" Oct 30 22:41:25.599324 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:25.599307 4615 scope.go:122] "RemoveContainer" containerID="20e5f9d8e3c95a7d00c3230bd70a692c4163e81202fff33a8c13662d819cac89" Oct 30 22:41:25.599576 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:41:25.599554 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20e5f9d8e3c95a7d00c3230bd70a692c4163e81202fff33a8c13662d819cac89\": container with ID starting with 20e5f9d8e3c95a7d00c3230bd70a692c4163e81202fff33a8c13662d819cac89 not found: ID does not exist" containerID="20e5f9d8e3c95a7d00c3230bd70a692c4163e81202fff33a8c13662d819cac89" Oct 30 22:41:25.599641 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:25.599585 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20e5f9d8e3c95a7d00c3230bd70a692c4163e81202fff33a8c13662d819cac89"} err="failed to get container status \"20e5f9d8e3c95a7d00c3230bd70a692c4163e81202fff33a8c13662d819cac89\": rpc error: code = NotFound desc = could not find container \"20e5f9d8e3c95a7d00c3230bd70a692c4163e81202fff33a8c13662d819cac89\": container with ID starting with 20e5f9d8e3c95a7d00c3230bd70a692c4163e81202fff33a8c13662d819cac89 not found: ID does not exist" Oct 30 22:41:25.599641 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:25.599603 4615 scope.go:122] "RemoveContainer" containerID="e7115c90845d4d1c482af2d4a436b6ca99857a4baa16ed6e91457ca4314e1dab" Oct 30 22:41:25.599847 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:41:25.599831 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7115c90845d4d1c482af2d4a436b6ca99857a4baa16ed6e91457ca4314e1dab\": container with ID starting with e7115c90845d4d1c482af2d4a436b6ca99857a4baa16ed6e91457ca4314e1dab not found: ID does not exist" containerID="e7115c90845d4d1c482af2d4a436b6ca99857a4baa16ed6e91457ca4314e1dab" Oct 30 22:41:25.599894 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:25.599853 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7115c90845d4d1c482af2d4a436b6ca99857a4baa16ed6e91457ca4314e1dab"} err="failed to get container status \"e7115c90845d4d1c482af2d4a436b6ca99857a4baa16ed6e91457ca4314e1dab\": rpc error: code = NotFound desc = could not find container \"e7115c90845d4d1c482af2d4a436b6ca99857a4baa16ed6e91457ca4314e1dab\": container with ID starting with e7115c90845d4d1c482af2d4a436b6ca99857a4baa16ed6e91457ca4314e1dab not found: ID does not exist" Oct 30 22:41:25.629547 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:25.629522 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc"] Oct 30 22:41:25.641020 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:25.640998 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-c00ac-predictor-689b7cb56b-dk5kc"] Oct 30 22:41:27.232762 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:27.232726 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d59524f5-bf80-4785-8038-5a93d6fead9b" path="/var/lib/kubelet/pods/d59524f5-bf80-4785-8038-5a93d6fead9b/volumes" Oct 30 22:41:29.763016 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:41:29.762972 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:41:31.499820 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:31.499785 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:41:31.691604 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:31.691572 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:41:34.436116 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:34.436079 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" podUID="9b7c3e3f-a09b-4eaf-91a9-f7809e028efd" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.42:8080: connect: connection refused" Oct 30 22:41:34.634413 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:34.634371 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:41:35.382399 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:35.382365 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:41:44.437111 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:44.437078 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" Oct 30 22:41:46.030400 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:46.030368 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:41:46.673090 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:46.673033 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:41:52.070369 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:52.070323 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:41:55.396375 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:55.396333 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:41:55.515326 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:41:55.515294 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:41:59.762146 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:41:59.762112 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:42:23.174809 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:42:23.174767 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:42:23.922929 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:42:23.922882 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:42:29.762523 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:42:29.762486 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:42:52.177969 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:42:52.177939 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:42:59.763011 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:42:59.762977 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:43:03.434393 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:03.434364 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:43:03.444594 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:03.444573 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:43:29.762846 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:43:29.762774 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:43:37.525931 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:37.525898 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:43:37.701931 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:37.701891 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:43:38.262198 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.262163 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:43:38.277542 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.277513 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw"] Oct 30 22:43:38.278012 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.277993 4615 cpu_manager.go:414] "containerMap: removing container" podUID="d59524f5-bf80-4785-8038-5a93d6fead9b" containerName="kserve-container" Oct 30 22:43:38.278102 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.278015 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d59524f5-bf80-4785-8038-5a93d6fead9b" containerName="kserve-container" Oct 30 22:43:38.278102 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.278032 4615 cpu_manager.go:414] "containerMap: removing container" podUID="d59524f5-bf80-4785-8038-5a93d6fead9b" containerName="kube-rbac-proxy" Oct 30 22:43:38.278102 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.278040 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d59524f5-bf80-4785-8038-5a93d6fead9b" containerName="kube-rbac-proxy" Oct 30 22:43:38.278206 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.278164 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="d59524f5-bf80-4785-8038-5a93d6fead9b" containerName="kserve-container" Oct 30 22:43:38.278206 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.278174 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="d59524f5-bf80-4785-8038-5a93d6fead9b" containerName="kube-rbac-proxy" Oct 30 22:43:38.281250 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.281235 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" Oct 30 22:43:38.297903 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.297884 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-cc765036c8\"/\"error-404-isvc-08e9a-predictor-serving-cert\"" Oct 30 22:43:38.309914 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.309894 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-cc765036c8\"/\"error-404-isvc-08e9a-kube-rbac-proxy-sar-config\"" Oct 30 22:43:38.310024 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.309918 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-cc765036c8\"/\"kube-root-ca.crt\"" Oct 30 22:43:38.311499 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.311470 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw"] Oct 30 22:43:38.399420 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.399391 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:43:38.412549 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.412512 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b64d9a50-7346-40dc-973b-5949ae6b977c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-08e9a-predictor-74d57bc954-r4ccw\" (UID: \"b64d9a50-7346-40dc-973b-5949ae6b977c\") " pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" Oct 30 22:43:38.412681 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.412561 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b64d9a50-7346-40dc-973b-5949ae6b977c-proxy-tls\") pod \"error-404-isvc-08e9a-predictor-74d57bc954-r4ccw\" (UID: \"b64d9a50-7346-40dc-973b-5949ae6b977c\") " pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" Oct 30 22:43:38.412681 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.412652 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgl2c\" (UniqueName: \"kubernetes.io/projected/b64d9a50-7346-40dc-973b-5949ae6b977c-kube-api-access-bgl2c\") pod \"error-404-isvc-08e9a-predictor-74d57bc954-r4ccw\" (UID: \"b64d9a50-7346-40dc-973b-5949ae6b977c\") " pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" Oct 30 22:43:38.514146 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.514026 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b64d9a50-7346-40dc-973b-5949ae6b977c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-08e9a-predictor-74d57bc954-r4ccw\" (UID: \"b64d9a50-7346-40dc-973b-5949ae6b977c\") " pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" Oct 30 22:43:38.514146 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.514117 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b64d9a50-7346-40dc-973b-5949ae6b977c-proxy-tls\") pod \"error-404-isvc-08e9a-predictor-74d57bc954-r4ccw\" (UID: \"b64d9a50-7346-40dc-973b-5949ae6b977c\") " pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" Oct 30 22:43:38.514337 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.514198 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgl2c\" (UniqueName: \"kubernetes.io/projected/b64d9a50-7346-40dc-973b-5949ae6b977c-kube-api-access-bgl2c\") pod \"error-404-isvc-08e9a-predictor-74d57bc954-r4ccw\" (UID: \"b64d9a50-7346-40dc-973b-5949ae6b977c\") " pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" Oct 30 22:43:38.514763 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.514740 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b64d9a50-7346-40dc-973b-5949ae6b977c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-08e9a-predictor-74d57bc954-r4ccw\" (UID: \"b64d9a50-7346-40dc-973b-5949ae6b977c\") " pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" Oct 30 22:43:38.516460 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.516443 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b64d9a50-7346-40dc-973b-5949ae6b977c-proxy-tls\") pod \"error-404-isvc-08e9a-predictor-74d57bc954-r4ccw\" (UID: \"b64d9a50-7346-40dc-973b-5949ae6b977c\") " pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" Oct 30 22:43:38.534965 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.534936 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgl2c\" (UniqueName: \"kubernetes.io/projected/b64d9a50-7346-40dc-973b-5949ae6b977c-kube-api-access-bgl2c\") pod \"error-404-isvc-08e9a-predictor-74d57bc954-r4ccw\" (UID: \"b64d9a50-7346-40dc-973b-5949ae6b977c\") " pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" Oct 30 22:43:38.595792 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.595765 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" Oct 30 22:43:38.901464 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:38.901437 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw"] Oct 30 22:43:38.902921 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:43:38.902891 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb64d9a50_7346_40dc_973b_5949ae6b977c.slice/crio-37efb2aa4956f65e93fc5f4e14711f0b4194f80eb4b9fb5c8cfac3b525212eff WatchSource:0}: Error finding container 37efb2aa4956f65e93fc5f4e14711f0b4194f80eb4b9fb5c8cfac3b525212eff: Status 404 returned error can't find the container with id 37efb2aa4956f65e93fc5f4e14711f0b4194f80eb4b9fb5c8cfac3b525212eff Oct 30 22:43:39.120004 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:39.119963 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" event={"ID":"b64d9a50-7346-40dc-973b-5949ae6b977c","Type":"ContainerStarted","Data":"2300a2ea1b82c16b165c22e632b46a983c9a9266dee38e7e062b1a769e420a31"} Oct 30 22:43:39.120207 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:39.120020 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" event={"ID":"b64d9a50-7346-40dc-973b-5949ae6b977c","Type":"ContainerStarted","Data":"700cf19ea9c52b85fd9a07a404a250e879b2c0ebf3e8b1163b1e82b65df98175"} Oct 30 22:43:39.120207 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:39.120155 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" Oct 30 22:43:39.120207 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:39.120177 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" event={"ID":"b64d9a50-7346-40dc-973b-5949ae6b977c","Type":"ContainerStarted","Data":"37efb2aa4956f65e93fc5f4e14711f0b4194f80eb4b9fb5c8cfac3b525212eff"} Oct 30 22:43:39.120207 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:39.120196 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" Oct 30 22:43:39.121518 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:39.121490 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" podUID="b64d9a50-7346-40dc-973b-5949ae6b977c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused" Oct 30 22:43:39.162289 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:39.162194 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" podStartSLOduration=1.162177844 podStartE2EDuration="1.162177844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:43:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:43:39.16153155 +0000 UTC m=+2136.077217862" watchObservedRunningTime="2026-10-30 22:43:39.162177844 +0000 UTC m=+2136.077864148" Oct 30 22:43:40.124648 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:40.124611 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" podUID="b64d9a50-7346-40dc-973b-5949ae6b977c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused" Oct 30 22:43:45.128954 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:45.128926 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" Oct 30 22:43:45.129439 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:45.129376 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" podUID="b64d9a50-7346-40dc-973b-5949ae6b977c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused" Oct 30 22:43:52.288514 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:52.288478 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:43:55.129674 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:43:55.129633 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" podUID="b64d9a50-7346-40dc-973b-5949ae6b977c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused" Oct 30 22:43:59.763847 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:43:59.763813 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:44:05.129415 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:44:05.129328 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" podUID="b64d9a50-7346-40dc-973b-5949ae6b977c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused" Oct 30 22:44:15.129392 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:44:15.129353 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" podUID="b64d9a50-7346-40dc-973b-5949ae6b977c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused" Oct 30 22:44:25.130269 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:44:25.130228 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" podUID="b64d9a50-7346-40dc-973b-5949ae6b977c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused" Oct 30 22:44:29.763712 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:44:29.763665 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:44:35.129943 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:44:35.129913 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" Oct 30 22:44:39.895899 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:44:39.895866 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:44:47.733304 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:44:47.733223 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:44:47.938098 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:44:47.938066 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:44:52.356704 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:44:52.356664 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:44:59.763035 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:44:59.762993 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:45:19.067414 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:45:19.067378 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:45:23.145816 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:45:23.145785 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:45:29.762904 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:45:29.762867 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:45:52.496948 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:45:52.496914 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:45:59.764985 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:45:59.764950 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:46:29.763761 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:46:29.763678 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:46:52.620456 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:46:52.620424 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:46:59.763352 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:46:59.763313 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:47:29.762566 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:47:29.762532 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:47:52.735413 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:47:52.735379 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:47:59.762982 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:47:59.762950 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:48:03.464136 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:48:03.464105 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:48:03.470073 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:48:03.470038 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:48:29.762292 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:48:29.762256 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:48:52.861483 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:48:52.861450 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:48:59.763276 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:48:59.763233 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:49:29.763104 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:49:29.763007 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:49:52.947756 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:49:52.947717 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:49:59.762480 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:49:59.762443 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:50:29.763358 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:50:29.763320 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:50:53.020181 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:50:53.020098 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:50:59.763039 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:50:59.763007 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:51:29.763192 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:51:29.763156 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:51:31.167884 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:51:31.167843 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:51:53.211363 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:51:53.211329 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:51:59.762955 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:51:59.762906 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:52:24.593606 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:52:24.593523 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:52:29.763010 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:52:29.762970 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:52:53.295770 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:52:53.295735 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:52:59.761409 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:52:59.761369 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:53:03.329831 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:03.329802 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw"] Oct 30 22:53:03.330272 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:03.330004 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" podUID="b64d9a50-7346-40dc-973b-5949ae6b977c" containerName="kserve-container" containerID="cri-o://700cf19ea9c52b85fd9a07a404a250e879b2c0ebf3e8b1163b1e82b65df98175" gracePeriod=30 Oct 30 22:53:03.330272 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:03.330024 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" podUID="b64d9a50-7346-40dc-973b-5949ae6b977c" containerName="kube-rbac-proxy" containerID="cri-o://2300a2ea1b82c16b165c22e632b46a983c9a9266dee38e7e062b1a769e420a31" gracePeriod=30 Oct 30 22:53:03.399733 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:03.399689 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:53:03.458587 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:03.458555 4615 generic.go:356] "Generic (PLEG): container finished" podID="b64d9a50-7346-40dc-973b-5949ae6b977c" containerID="2300a2ea1b82c16b165c22e632b46a983c9a9266dee38e7e062b1a769e420a31" exitCode=0 Oct 30 22:53:03.458721 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:03.458630 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" event={"ID":"b64d9a50-7346-40dc-973b-5949ae6b977c","Type":"ContainerDied","Data":"2300a2ea1b82c16b165c22e632b46a983c9a9266dee38e7e062b1a769e420a31"} Oct 30 22:53:03.493250 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:03.493218 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:53:03.495571 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:03.495548 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:53:03.522704 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:03.522680 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:53:05.125725 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:05.125690 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" podUID="b64d9a50-7346-40dc-973b-5949ae6b977c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.44:8643/healthz\": dial tcp 10.134.0.44:8643: connect: connection refused" Oct 30 22:53:05.130030 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:05.130009 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" podUID="b64d9a50-7346-40dc-973b-5949ae6b977c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.44:8080: connect: connection refused" Oct 30 22:53:06.475772 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:06.475743 4615 generic.go:356] "Generic (PLEG): container finished" podID="b64d9a50-7346-40dc-973b-5949ae6b977c" containerID="700cf19ea9c52b85fd9a07a404a250e879b2c0ebf3e8b1163b1e82b65df98175" exitCode=0 Oct 30 22:53:06.476120 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:06.475814 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" event={"ID":"b64d9a50-7346-40dc-973b-5949ae6b977c","Type":"ContainerDied","Data":"700cf19ea9c52b85fd9a07a404a250e879b2c0ebf3e8b1163b1e82b65df98175"} Oct 30 22:53:06.867389 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:06.867366 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" Oct 30 22:53:06.971700 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:06.971671 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b64d9a50-7346-40dc-973b-5949ae6b977c-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b64d9a50-7346-40dc-973b-5949ae6b977c-kube-rbac-proxy-sar-config\") pod \"b64d9a50-7346-40dc-973b-5949ae6b977c\" (UID: \"b64d9a50-7346-40dc-973b-5949ae6b977c\") " Oct 30 22:53:06.971858 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:06.971738 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b64d9a50-7346-40dc-973b-5949ae6b977c-proxy-tls\" (UniqueName: \"kubernetes.io/secret/b64d9a50-7346-40dc-973b-5949ae6b977c-proxy-tls\") pod \"b64d9a50-7346-40dc-973b-5949ae6b977c\" (UID: \"b64d9a50-7346-40dc-973b-5949ae6b977c\") " Oct 30 22:53:06.971858 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:06.971762 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b64d9a50-7346-40dc-973b-5949ae6b977c-kube-api-access-bgl2c\" (UniqueName: \"kubernetes.io/projected/b64d9a50-7346-40dc-973b-5949ae6b977c-kube-api-access-bgl2c\") pod \"b64d9a50-7346-40dc-973b-5949ae6b977c\" (UID: \"b64d9a50-7346-40dc-973b-5949ae6b977c\") " Oct 30 22:53:06.972107 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:06.972082 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b64d9a50-7346-40dc-973b-5949ae6b977c-kube-rbac-proxy-sar-config" pod "b64d9a50-7346-40dc-973b-5949ae6b977c" (UID: "b64d9a50-7346-40dc-973b-5949ae6b977c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:53:06.973747 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:06.973726 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b64d9a50-7346-40dc-973b-5949ae6b977c-proxy-tls" pod "b64d9a50-7346-40dc-973b-5949ae6b977c" (UID: "b64d9a50-7346-40dc-973b-5949ae6b977c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:53:06.973806 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:06.973776 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b64d9a50-7346-40dc-973b-5949ae6b977c-kube-api-access-bgl2c" pod "b64d9a50-7346-40dc-973b-5949ae6b977c" (UID: "b64d9a50-7346-40dc-973b-5949ae6b977c"). InnerVolumeSpecName "kube-api-access-bgl2c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:53:07.072276 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:07.072206 4615 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b64d9a50-7346-40dc-973b-5949ae6b977c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:53:07.072276 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:07.072232 4615 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b64d9a50-7346-40dc-973b-5949ae6b977c-proxy-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:53:07.072276 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:07.072243 4615 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgl2c\" (UniqueName: \"kubernetes.io/projected/b64d9a50-7346-40dc-973b-5949ae6b977c-kube-api-access-bgl2c\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:53:07.481744 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:07.481707 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" event={"ID":"b64d9a50-7346-40dc-973b-5949ae6b977c","Type":"ContainerDied","Data":"37efb2aa4956f65e93fc5f4e14711f0b4194f80eb4b9fb5c8cfac3b525212eff"} Oct 30 22:53:07.482253 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:07.481760 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw" Oct 30 22:53:07.482253 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:07.481798 4615 scope.go:122] "RemoveContainer" containerID="2300a2ea1b82c16b165c22e632b46a983c9a9266dee38e7e062b1a769e420a31" Oct 30 22:53:07.489928 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:07.489910 4615 scope.go:122] "RemoveContainer" containerID="700cf19ea9c52b85fd9a07a404a250e879b2c0ebf3e8b1163b1e82b65df98175" Oct 30 22:53:07.531752 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:07.531729 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw"] Oct 30 22:53:07.543389 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:07.543354 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-cc765036c8/error-404-isvc-08e9a-predictor-74d57bc954-r4ccw"] Oct 30 22:53:08.198829 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:08.198792 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:53:09.234485 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:09.234455 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b64d9a50-7346-40dc-973b-5949ae6b977c" path="/var/lib/kubelet/pods/b64d9a50-7346-40dc-973b-5949ae6b977c/volumes" Oct 30 22:53:13.144988 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:13.144957 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh"] Oct 30 22:53:13.145458 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:13.145163 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" podUID="876dbfb5-875f-4336-8247-982a6ac58d37" containerName="ensemble-graph-83e67" containerID="cri-o://aab63736d527416674e2efea4b21b2a5ca63c3652d37460f3727e17728fb5978" gracePeriod=30 Oct 30 22:53:13.156600 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:13.156573 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq"] Oct 30 22:53:13.156807 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:13.156774 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" podUID="1bf494db-f7a0-4126-96aa-3819315dc2a0" containerName="ensemble-graph-c1ed1" containerID="cri-o://a51d3361cd5c4863518ca90e512b473014c7632f2001066e407a4f0d79102de5" gracePeriod=30 Oct 30 22:53:13.165344 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:13.165324 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs"] Oct 30 22:53:13.165506 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:13.165489 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" podUID="a58e4278-3be4-415f-90f2-6a3c42afbb6f" containerName="splitter-graph-c00ac" containerID="cri-o://d5734dbfea3666605de0e02a2f85b4a485b780e5a633f1b6c71600293325e180" gracePeriod=30 Oct 30 22:53:13.278747 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:13.278716 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:53:14.752214 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:14.752174 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" podUID="1bf494db-f7a0-4126-96aa-3819315dc2a0" containerName="ensemble-graph-c1ed1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:53:16.131097 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:16.131035 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" podUID="876dbfb5-875f-4336-8247-982a6ac58d37" containerName="ensemble-graph-83e67" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:53:17.527437 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:17.527401 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" podUID="a58e4278-3be4-415f-90f2-6a3c42afbb6f" containerName="splitter-graph-c00ac" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:53:19.752097 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:19.752036 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" podUID="1bf494db-f7a0-4126-96aa-3819315dc2a0" containerName="ensemble-graph-c1ed1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:53:21.130985 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:21.130941 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" podUID="876dbfb5-875f-4336-8247-982a6ac58d37" containerName="ensemble-graph-83e67" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:53:22.527841 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:22.527797 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" podUID="a58e4278-3be4-415f-90f2-6a3c42afbb6f" containerName="splitter-graph-c00ac" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:53:24.752383 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:24.752338 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" podUID="1bf494db-f7a0-4126-96aa-3819315dc2a0" containerName="ensemble-graph-c1ed1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:53:24.752847 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:24.752496 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" Oct 30 22:53:26.130442 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:26.130401 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" podUID="876dbfb5-875f-4336-8247-982a6ac58d37" containerName="ensemble-graph-83e67" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:53:26.130863 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:26.130526 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" Oct 30 22:53:27.528441 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:27.528399 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" podUID="a58e4278-3be4-415f-90f2-6a3c42afbb6f" containerName="splitter-graph-c00ac" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:53:27.528849 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:27.528503 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" Oct 30 22:53:29.751946 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:29.751909 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" podUID="1bf494db-f7a0-4126-96aa-3819315dc2a0" containerName="ensemble-graph-c1ed1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:53:29.761777 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:53:29.761746 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:53:31.130950 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:31.130913 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" podUID="876dbfb5-875f-4336-8247-982a6ac58d37" containerName="ensemble-graph-83e67" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:53:32.527761 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:32.527712 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" podUID="a58e4278-3be4-415f-90f2-6a3c42afbb6f" containerName="splitter-graph-c00ac" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:53:34.751991 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:34.751953 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" podUID="1bf494db-f7a0-4126-96aa-3819315dc2a0" containerName="ensemble-graph-c1ed1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:53:36.130334 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:36.130294 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" podUID="876dbfb5-875f-4336-8247-982a6ac58d37" containerName="ensemble-graph-83e67" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:53:37.528533 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:37.528496 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" podUID="a58e4278-3be4-415f-90f2-6a3c42afbb6f" containerName="splitter-graph-c00ac" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:53:39.752395 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:39.752356 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" podUID="1bf494db-f7a0-4126-96aa-3819315dc2a0" containerName="ensemble-graph-c1ed1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:53:41.131119 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:41.131083 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" podUID="876dbfb5-875f-4336-8247-982a6ac58d37" containerName="ensemble-graph-83e67" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:53:42.528128 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:42.528089 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" podUID="a58e4278-3be4-415f-90f2-6a3c42afbb6f" containerName="splitter-graph-c00ac" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:53:43.342870 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.342843 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" Oct 30 22:53:43.345627 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.345605 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" Oct 30 22:53:43.348674 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.348656 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" Oct 30 22:53:43.475228 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.475145 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/876dbfb5-875f-4336-8247-982a6ac58d37-proxy-tls\" (UniqueName: \"kubernetes.io/secret/876dbfb5-875f-4336-8247-982a6ac58d37-proxy-tls\") pod \"876dbfb5-875f-4336-8247-982a6ac58d37\" (UID: \"876dbfb5-875f-4336-8247-982a6ac58d37\") " Oct 30 22:53:43.475228 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.475183 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/876dbfb5-875f-4336-8247-982a6ac58d37-openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/876dbfb5-875f-4336-8247-982a6ac58d37-openshift-service-ca-bundle\") pod \"876dbfb5-875f-4336-8247-982a6ac58d37\" (UID: \"876dbfb5-875f-4336-8247-982a6ac58d37\") " Oct 30 22:53:43.475454 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.475242 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a58e4278-3be4-415f-90f2-6a3c42afbb6f-openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a58e4278-3be4-415f-90f2-6a3c42afbb6f-openshift-service-ca-bundle\") pod \"a58e4278-3be4-415f-90f2-6a3c42afbb6f\" (UID: \"a58e4278-3be4-415f-90f2-6a3c42afbb6f\") " Oct 30 22:53:43.475454 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.475284 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1bf494db-f7a0-4126-96aa-3819315dc2a0-openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bf494db-f7a0-4126-96aa-3819315dc2a0-openshift-service-ca-bundle\") pod \"1bf494db-f7a0-4126-96aa-3819315dc2a0\" (UID: \"1bf494db-f7a0-4126-96aa-3819315dc2a0\") " Oct 30 22:53:43.475454 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.475329 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1bf494db-f7a0-4126-96aa-3819315dc2a0-proxy-tls\" (UniqueName: \"kubernetes.io/secret/1bf494db-f7a0-4126-96aa-3819315dc2a0-proxy-tls\") pod \"1bf494db-f7a0-4126-96aa-3819315dc2a0\" (UID: \"1bf494db-f7a0-4126-96aa-3819315dc2a0\") " Oct 30 22:53:43.475454 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.475417 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a58e4278-3be4-415f-90f2-6a3c42afbb6f-proxy-tls\" (UniqueName: \"kubernetes.io/secret/a58e4278-3be4-415f-90f2-6a3c42afbb6f-proxy-tls\") pod \"a58e4278-3be4-415f-90f2-6a3c42afbb6f\" (UID: \"a58e4278-3be4-415f-90f2-6a3c42afbb6f\") " Oct 30 22:53:43.475663 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.475633 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/876dbfb5-875f-4336-8247-982a6ac58d37-openshift-service-ca-bundle" pod "876dbfb5-875f-4336-8247-982a6ac58d37" (UID: "876dbfb5-875f-4336-8247-982a6ac58d37"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:53:43.475734 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.475636 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a58e4278-3be4-415f-90f2-6a3c42afbb6f-openshift-service-ca-bundle" pod "a58e4278-3be4-415f-90f2-6a3c42afbb6f" (UID: "a58e4278-3be4-415f-90f2-6a3c42afbb6f"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:53:43.475896 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.475871 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bf494db-f7a0-4126-96aa-3819315dc2a0-openshift-service-ca-bundle" pod "1bf494db-f7a0-4126-96aa-3819315dc2a0" (UID: "1bf494db-f7a0-4126-96aa-3819315dc2a0"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:53:43.477685 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.477657 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/876dbfb5-875f-4336-8247-982a6ac58d37-proxy-tls" pod "876dbfb5-875f-4336-8247-982a6ac58d37" (UID: "876dbfb5-875f-4336-8247-982a6ac58d37"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:53:43.477685 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.477669 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a58e4278-3be4-415f-90f2-6a3c42afbb6f-proxy-tls" pod "a58e4278-3be4-415f-90f2-6a3c42afbb6f" (UID: "a58e4278-3be4-415f-90f2-6a3c42afbb6f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:53:43.477810 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.477705 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bf494db-f7a0-4126-96aa-3819315dc2a0-proxy-tls" pod "1bf494db-f7a0-4126-96aa-3819315dc2a0" (UID: "1bf494db-f7a0-4126-96aa-3819315dc2a0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:53:43.576220 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.576188 4615 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a58e4278-3be4-415f-90f2-6a3c42afbb6f-proxy-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:53:43.576220 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.576213 4615 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/876dbfb5-875f-4336-8247-982a6ac58d37-proxy-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:53:43.576220 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.576222 4615 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/876dbfb5-875f-4336-8247-982a6ac58d37-openshift-service-ca-bundle\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:53:43.576634 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.576232 4615 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a58e4278-3be4-415f-90f2-6a3c42afbb6f-openshift-service-ca-bundle\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:53:43.576634 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.576240 4615 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1bf494db-f7a0-4126-96aa-3819315dc2a0-openshift-service-ca-bundle\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:53:43.576634 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.576249 4615 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1bf494db-f7a0-4126-96aa-3819315dc2a0-proxy-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:53:43.628691 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.628658 4615 generic.go:356] "Generic (PLEG): container finished" podID="876dbfb5-875f-4336-8247-982a6ac58d37" containerID="aab63736d527416674e2efea4b21b2a5ca63c3652d37460f3727e17728fb5978" exitCode=0 Oct 30 22:53:43.628848 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.628704 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" Oct 30 22:53:43.628848 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.628702 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" event={"ID":"876dbfb5-875f-4336-8247-982a6ac58d37","Type":"ContainerDied","Data":"aab63736d527416674e2efea4b21b2a5ca63c3652d37460f3727e17728fb5978"} Oct 30 22:53:43.628947 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.628878 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh" event={"ID":"876dbfb5-875f-4336-8247-982a6ac58d37","Type":"ContainerDied","Data":"59a17a6bfe87d48c37435a19703acc1751bbf4501fad69a6da6530090ca52ef9"} Oct 30 22:53:43.628947 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.628914 4615 scope.go:122] "RemoveContainer" containerID="aab63736d527416674e2efea4b21b2a5ca63c3652d37460f3727e17728fb5978" Oct 30 22:53:43.630090 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.629986 4615 generic.go:356] "Generic (PLEG): container finished" podID="1bf494db-f7a0-4126-96aa-3819315dc2a0" containerID="a51d3361cd5c4863518ca90e512b473014c7632f2001066e407a4f0d79102de5" exitCode=137 Oct 30 22:53:43.630090 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.630060 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" Oct 30 22:53:43.630090 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.630069 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" event={"ID":"1bf494db-f7a0-4126-96aa-3819315dc2a0","Type":"ContainerDied","Data":"a51d3361cd5c4863518ca90e512b473014c7632f2001066e407a4f0d79102de5"} Oct 30 22:53:43.630287 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.630122 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq" event={"ID":"1bf494db-f7a0-4126-96aa-3819315dc2a0","Type":"ContainerDied","Data":"e71342779312362690842f5cbda354451129fc5c97efb171cb07bf330fe33b21"} Oct 30 22:53:43.631310 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.631286 4615 generic.go:356] "Generic (PLEG): container finished" podID="a58e4278-3be4-415f-90f2-6a3c42afbb6f" containerID="d5734dbfea3666605de0e02a2f85b4a485b780e5a633f1b6c71600293325e180" exitCode=0 Oct 30 22:53:43.631422 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.631408 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" event={"ID":"a58e4278-3be4-415f-90f2-6a3c42afbb6f","Type":"ContainerDied","Data":"d5734dbfea3666605de0e02a2f85b4a485b780e5a633f1b6c71600293325e180"} Oct 30 22:53:43.631470 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.631446 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" event={"ID":"a58e4278-3be4-415f-90f2-6a3c42afbb6f","Type":"ContainerDied","Data":"d6866bbe540d907c95198da53b1bfe70ee8d9b322eff414c4102e4ce7cade836"} Oct 30 22:53:43.631521 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.631508 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs" Oct 30 22:53:43.643066 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.640234 4615 scope.go:122] "RemoveContainer" containerID="aab63736d527416674e2efea4b21b2a5ca63c3652d37460f3727e17728fb5978" Oct 30 22:53:43.645065 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:53:43.643504 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aab63736d527416674e2efea4b21b2a5ca63c3652d37460f3727e17728fb5978\": container with ID starting with aab63736d527416674e2efea4b21b2a5ca63c3652d37460f3727e17728fb5978 not found: ID does not exist" containerID="aab63736d527416674e2efea4b21b2a5ca63c3652d37460f3727e17728fb5978" Oct 30 22:53:43.645065 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.643542 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aab63736d527416674e2efea4b21b2a5ca63c3652d37460f3727e17728fb5978"} err="failed to get container status \"aab63736d527416674e2efea4b21b2a5ca63c3652d37460f3727e17728fb5978\": rpc error: code = NotFound desc = could not find container \"aab63736d527416674e2efea4b21b2a5ca63c3652d37460f3727e17728fb5978\": container with ID starting with aab63736d527416674e2efea4b21b2a5ca63c3652d37460f3727e17728fb5978 not found: ID does not exist" Oct 30 22:53:43.645065 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.643567 4615 scope.go:122] "RemoveContainer" containerID="a51d3361cd5c4863518ca90e512b473014c7632f2001066e407a4f0d79102de5" Oct 30 22:53:43.655064 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.655028 4615 scope.go:122] "RemoveContainer" containerID="a51d3361cd5c4863518ca90e512b473014c7632f2001066e407a4f0d79102de5" Oct 30 22:53:43.655323 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:53:43.655294 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a51d3361cd5c4863518ca90e512b473014c7632f2001066e407a4f0d79102de5\": container with ID starting with a51d3361cd5c4863518ca90e512b473014c7632f2001066e407a4f0d79102de5 not found: ID does not exist" containerID="a51d3361cd5c4863518ca90e512b473014c7632f2001066e407a4f0d79102de5" Oct 30 22:53:43.655397 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.655325 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a51d3361cd5c4863518ca90e512b473014c7632f2001066e407a4f0d79102de5"} err="failed to get container status \"a51d3361cd5c4863518ca90e512b473014c7632f2001066e407a4f0d79102de5\": rpc error: code = NotFound desc = could not find container \"a51d3361cd5c4863518ca90e512b473014c7632f2001066e407a4f0d79102de5\": container with ID starting with a51d3361cd5c4863518ca90e512b473014c7632f2001066e407a4f0d79102de5 not found: ID does not exist" Oct 30 22:53:43.655397 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.655347 4615 scope.go:122] "RemoveContainer" containerID="d5734dbfea3666605de0e02a2f85b4a485b780e5a633f1b6c71600293325e180" Oct 30 22:53:43.663166 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.663149 4615 scope.go:122] "RemoveContainer" containerID="d5734dbfea3666605de0e02a2f85b4a485b780e5a633f1b6c71600293325e180" Oct 30 22:53:43.663385 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:53:43.663369 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5734dbfea3666605de0e02a2f85b4a485b780e5a633f1b6c71600293325e180\": container with ID starting with d5734dbfea3666605de0e02a2f85b4a485b780e5a633f1b6c71600293325e180 not found: ID does not exist" containerID="d5734dbfea3666605de0e02a2f85b4a485b780e5a633f1b6c71600293325e180" Oct 30 22:53:43.663433 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.663390 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5734dbfea3666605de0e02a2f85b4a485b780e5a633f1b6c71600293325e180"} err="failed to get container status \"d5734dbfea3666605de0e02a2f85b4a485b780e5a633f1b6c71600293325e180\": rpc error: code = NotFound desc = could not find container \"d5734dbfea3666605de0e02a2f85b4a485b780e5a633f1b6c71600293325e180\": container with ID starting with d5734dbfea3666605de0e02a2f85b4a485b780e5a633f1b6c71600293325e180 not found: ID does not exist" Oct 30 22:53:43.707605 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.707579 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh"] Oct 30 22:53:43.711616 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.711597 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-cc765036c8/ensemble-graph-83e67-68c5d8576-j9hdh"] Oct 30 22:53:43.743530 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.743470 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs"] Oct 30 22:53:43.747116 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.747096 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-cc765036c8/splitter-graph-c00ac-fdb955b67-67rgs"] Oct 30 22:53:43.779944 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.779915 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq"] Oct 30 22:53:43.784011 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:43.783990 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-cc765036c8/ensemble-graph-c1ed1-75cfc869c7-mgkzq"] Oct 30 22:53:45.233374 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:45.233340 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1bf494db-f7a0-4126-96aa-3819315dc2a0" path="/var/lib/kubelet/pods/1bf494db-f7a0-4126-96aa-3819315dc2a0/volumes" Oct 30 22:53:45.233753 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:45.233663 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="876dbfb5-875f-4336-8247-982a6ac58d37" path="/var/lib/kubelet/pods/876dbfb5-875f-4336-8247-982a6ac58d37/volumes" Oct 30 22:53:45.233950 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:45.233938 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a58e4278-3be4-415f-90f2-6a3c42afbb6f" path="/var/lib/kubelet/pods/a58e4278-3be4-415f-90f2-6a3c42afbb6f/volumes" Oct 30 22:53:50.497740 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:50.497661 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:53:50.614298 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:50.614267 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:53:52.697243 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:52.697204 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:53:53.405369 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:53.405335 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:53:58.179167 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:58.179130 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:53:58.289777 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:53:58.289748 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:53:59.762327 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:53:59.762286 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:54:29.762490 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:54:29.762452 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:54:53.508492 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:54:53.508455 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:54:59.761290 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:54:59.761253 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:55:29.762157 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:55:29.762070 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:55:53.632707 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:55:53.632673 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:55:59.762758 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:55:59.762722 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:56:29.762125 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:56:29.762084 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:56:53.618071 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:56:53.617948 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:56:53.732291 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:56:53.732237 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:56:59.763199 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:56:59.763156 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:57:29.761945 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:57:29.761906 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:57:53.893430 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:57:53.893398 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:57:59.762214 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:57:59.762174 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:58:03.520249 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:03.520217 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:58:03.520778 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:03.520712 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wqj4x_95637308-9cdc-4a32-b573-70e1b4cff546/ovn-acl-logging/0.log" Oct 30 22:58:16.007894 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:16.007856 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt"] Oct 30 22:58:16.008381 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:16.008147 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" podUID="9b7c3e3f-a09b-4eaf-91a9-f7809e028efd" containerName="kserve-container" containerID="cri-o://888cd75691c5d9471fb3fac3edd126e568ac4590b9c3d96740b5c29e7ce65be9" gracePeriod=30 Oct 30 22:58:16.008381 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:16.008163 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" podUID="9b7c3e3f-a09b-4eaf-91a9-f7809e028efd" containerName="kube-rbac-proxy" containerID="cri-o://188058239a776b6f3078a2fe3f631393d0d131b28a4325d7ceb89eb7b8a971ff" gracePeriod=30 Oct 30 22:58:16.077528 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:16.077489 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:58:16.208800 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:16.208766 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:58:16.704749 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:16.704713 4615 generic.go:356] "Generic (PLEG): container finished" podID="9b7c3e3f-a09b-4eaf-91a9-f7809e028efd" containerID="188058239a776b6f3078a2fe3f631393d0d131b28a4325d7ceb89eb7b8a971ff" exitCode=0 Oct 30 22:58:16.704936 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:16.704760 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" event={"ID":"9b7c3e3f-a09b-4eaf-91a9-f7809e028efd","Type":"ContainerDied","Data":"188058239a776b6f3078a2fe3f631393d0d131b28a4325d7ceb89eb7b8a971ff"} Oct 30 22:58:19.353767 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:19.353741 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" Oct 30 22:58:19.419525 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:19.419492 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9b7c3e3f-a09b-4eaf-91a9-f7809e028efd-proxy-tls\" (UniqueName: \"kubernetes.io/secret/9b7c3e3f-a09b-4eaf-91a9-f7809e028efd-proxy-tls\") pod \"9b7c3e3f-a09b-4eaf-91a9-f7809e028efd\" (UID: \"9b7c3e3f-a09b-4eaf-91a9-f7809e028efd\") " Oct 30 22:58:19.419692 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:19.419559 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9b7c3e3f-a09b-4eaf-91a9-f7809e028efd-kube-api-access-h9wq9\" (UniqueName: \"kubernetes.io/projected/9b7c3e3f-a09b-4eaf-91a9-f7809e028efd-kube-api-access-h9wq9\") pod \"9b7c3e3f-a09b-4eaf-91a9-f7809e028efd\" (UID: \"9b7c3e3f-a09b-4eaf-91a9-f7809e028efd\") " Oct 30 22:58:19.419692 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:19.419582 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9b7c3e3f-a09b-4eaf-91a9-f7809e028efd-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9b7c3e3f-a09b-4eaf-91a9-f7809e028efd-kube-rbac-proxy-sar-config\") pod \"9b7c3e3f-a09b-4eaf-91a9-f7809e028efd\" (UID: \"9b7c3e3f-a09b-4eaf-91a9-f7809e028efd\") " Oct 30 22:58:19.420017 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:19.419992 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b7c3e3f-a09b-4eaf-91a9-f7809e028efd-kube-rbac-proxy-sar-config" pod "9b7c3e3f-a09b-4eaf-91a9-f7809e028efd" (UID: "9b7c3e3f-a09b-4eaf-91a9-f7809e028efd"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:58:19.421462 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:19.421431 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b7c3e3f-a09b-4eaf-91a9-f7809e028efd-proxy-tls" pod "9b7c3e3f-a09b-4eaf-91a9-f7809e028efd" (UID: "9b7c3e3f-a09b-4eaf-91a9-f7809e028efd"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:58:19.421568 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:19.421525 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b7c3e3f-a09b-4eaf-91a9-f7809e028efd-kube-api-access-h9wq9" pod "9b7c3e3f-a09b-4eaf-91a9-f7809e028efd" (UID: "9b7c3e3f-a09b-4eaf-91a9-f7809e028efd"). InnerVolumeSpecName "kube-api-access-h9wq9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 22:58:19.520338 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:19.520308 4615 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9wq9\" (UniqueName: \"kubernetes.io/projected/9b7c3e3f-a09b-4eaf-91a9-f7809e028efd-kube-api-access-h9wq9\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:58:19.520338 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:19.520336 4615 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/9b7c3e3f-a09b-4eaf-91a9-f7809e028efd-kube-rbac-proxy-sar-config\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:58:19.520338 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:19.520345 4615 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9b7c3e3f-a09b-4eaf-91a9-f7809e028efd-proxy-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:58:19.717574 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:19.717487 4615 generic.go:356] "Generic (PLEG): container finished" podID="9b7c3e3f-a09b-4eaf-91a9-f7809e028efd" containerID="888cd75691c5d9471fb3fac3edd126e568ac4590b9c3d96740b5c29e7ce65be9" exitCode=0 Oct 30 22:58:19.717574 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:19.717549 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" Oct 30 22:58:19.717791 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:19.717569 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" event={"ID":"9b7c3e3f-a09b-4eaf-91a9-f7809e028efd","Type":"ContainerDied","Data":"888cd75691c5d9471fb3fac3edd126e568ac4590b9c3d96740b5c29e7ce65be9"} Oct 30 22:58:19.717791 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:19.717629 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt" event={"ID":"9b7c3e3f-a09b-4eaf-91a9-f7809e028efd","Type":"ContainerDied","Data":"26ffbc52d9af20f2ec66eaa4977ae2295eba38202b77e6634ef7fab750bc4a2f"} Oct 30 22:58:19.717791 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:19.717657 4615 scope.go:122] "RemoveContainer" containerID="188058239a776b6f3078a2fe3f631393d0d131b28a4325d7ceb89eb7b8a971ff" Oct 30 22:58:19.726102 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:19.726084 4615 scope.go:122] "RemoveContainer" containerID="888cd75691c5d9471fb3fac3edd126e568ac4590b9c3d96740b5c29e7ce65be9" Oct 30 22:58:19.733033 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:19.733013 4615 scope.go:122] "RemoveContainer" containerID="188058239a776b6f3078a2fe3f631393d0d131b28a4325d7ceb89eb7b8a971ff" Oct 30 22:58:19.733724 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:58:19.733706 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"188058239a776b6f3078a2fe3f631393d0d131b28a4325d7ceb89eb7b8a971ff\": container with ID starting with 188058239a776b6f3078a2fe3f631393d0d131b28a4325d7ceb89eb7b8a971ff not found: ID does not exist" containerID="188058239a776b6f3078a2fe3f631393d0d131b28a4325d7ceb89eb7b8a971ff" Oct 30 22:58:19.733805 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:19.733736 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"188058239a776b6f3078a2fe3f631393d0d131b28a4325d7ceb89eb7b8a971ff"} err="failed to get container status \"188058239a776b6f3078a2fe3f631393d0d131b28a4325d7ceb89eb7b8a971ff\": rpc error: code = NotFound desc = could not find container \"188058239a776b6f3078a2fe3f631393d0d131b28a4325d7ceb89eb7b8a971ff\": container with ID starting with 188058239a776b6f3078a2fe3f631393d0d131b28a4325d7ceb89eb7b8a971ff not found: ID does not exist" Oct 30 22:58:19.733805 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:19.733758 4615 scope.go:122] "RemoveContainer" containerID="888cd75691c5d9471fb3fac3edd126e568ac4590b9c3d96740b5c29e7ce65be9" Oct 30 22:58:19.734009 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:58:19.733992 4615 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"888cd75691c5d9471fb3fac3edd126e568ac4590b9c3d96740b5c29e7ce65be9\": container with ID starting with 888cd75691c5d9471fb3fac3edd126e568ac4590b9c3d96740b5c29e7ce65be9 not found: ID does not exist" containerID="888cd75691c5d9471fb3fac3edd126e568ac4590b9c3d96740b5c29e7ce65be9" Oct 30 22:58:19.734074 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:19.734016 4615 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"888cd75691c5d9471fb3fac3edd126e568ac4590b9c3d96740b5c29e7ce65be9"} err="failed to get container status \"888cd75691c5d9471fb3fac3edd126e568ac4590b9c3d96740b5c29e7ce65be9\": rpc error: code = NotFound desc = could not find container \"888cd75691c5d9471fb3fac3edd126e568ac4590b9c3d96740b5c29e7ce65be9\": container with ID starting with 888cd75691c5d9471fb3fac3edd126e568ac4590b9c3d96740b5c29e7ce65be9 not found: ID does not exist" Oct 30 22:58:19.768057 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:19.768022 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt"] Oct 30 22:58:19.769982 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:19.769962 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-1b4518b221/error-404-isvc-504c0-predictor-f6b97589f-72rzt"] Oct 30 22:58:20.874615 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:20.874586 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:58:21.232688 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:21.232652 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9b7c3e3f-a09b-4eaf-91a9-f7809e028efd" path="/var/lib/kubelet/pods/9b7c3e3f-a09b-4eaf-91a9-f7809e028efd/volumes" Oct 30 22:58:22.495867 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:22.495832 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:58:22.635278 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:22.635245 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:58:25.824887 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:25.824851 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r"] Oct 30 22:58:25.825394 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:25.825112 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" podUID="490439c0-06df-43be-80ce-60586dd461c9" containerName="sequence-graph-15dee" containerID="cri-o://efc34ce3d208a8eb2e342483364956a1acc3483e07df0de2911ac96b8f788f88" gracePeriod=30 Oct 30 22:58:25.836652 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:25.836626 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh"] Oct 30 22:58:25.836861 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:25.836839 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" podUID="dd617f4a-e247-485e-85c7-83e715540f25" containerName="sequence-graph-4ace4" containerID="cri-o://b8667b84ee8a2b5848996e446102f03d58392555d723f28dd53843774a60eec9" gracePeriod=30 Oct 30 22:58:25.837731 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:25.837711 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t"] Oct 30 22:58:25.837904 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:25.837885 4615 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" podUID="241c8943-6710-48e5-b281-454b94b970be" containerName="sequence-graph-94a5a" containerID="cri-o://cf2ebea75ca962372265657be43ee22349e25b04e4992fd628487c637433a4f8" gracePeriod=30 Oct 30 22:58:25.947052 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:25.947013 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:58:26.326590 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:26.326551 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" podUID="490439c0-06df-43be-80ce-60586dd461c9" containerName="sequence-graph-15dee" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:58:27.605259 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:27.605223 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" podUID="241c8943-6710-48e5-b281-454b94b970be" containerName="sequence-graph-94a5a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:58:29.761892 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:58:29.761855 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:58:29.954624 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:29.954585 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" podUID="dd617f4a-e247-485e-85c7-83e715540f25" containerName="sequence-graph-4ace4" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:58:31.327184 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:31.327144 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" podUID="490439c0-06df-43be-80ce-60586dd461c9" containerName="sequence-graph-15dee" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:58:32.605284 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:32.605248 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" podUID="241c8943-6710-48e5-b281-454b94b970be" containerName="sequence-graph-94a5a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:58:34.954439 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:34.954398 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" podUID="dd617f4a-e247-485e-85c7-83e715540f25" containerName="sequence-graph-4ace4" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:58:36.327006 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:36.326967 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" podUID="490439c0-06df-43be-80ce-60586dd461c9" containerName="sequence-graph-15dee" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:58:36.327404 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:36.327092 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" Oct 30 22:58:37.604516 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:37.604476 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" podUID="241c8943-6710-48e5-b281-454b94b970be" containerName="sequence-graph-94a5a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:58:37.604974 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:37.604581 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" Oct 30 22:58:39.953881 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:39.953841 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" podUID="dd617f4a-e247-485e-85c7-83e715540f25" containerName="sequence-graph-4ace4" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:58:39.954295 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:39.953964 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" Oct 30 22:58:41.326769 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:41.326736 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" podUID="490439c0-06df-43be-80ce-60586dd461c9" containerName="sequence-graph-15dee" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:58:42.604772 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:42.604724 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" podUID="241c8943-6710-48e5-b281-454b94b970be" containerName="sequence-graph-94a5a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:58:43.195643 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:43.195612 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-15dee-557b5b7697-lmt2r_490439c0-06df-43be-80ce-60586dd461c9/sequence-graph-15dee/0.log" Oct 30 22:58:43.298538 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:43.298510 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-4ace4-86496f9c9-lqblh_dd617f4a-e247-485e-85c7-83e715540f25/sequence-graph-4ace4/0.log" Oct 30 22:58:43.372483 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:43.372453 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-94a5a-5bcd4597f4-krr2t_241c8943-6710-48e5-b281-454b94b970be/sequence-graph-94a5a/0.log" Oct 30 22:58:44.871579 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:44.871553 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-15dee-557b5b7697-lmt2r_490439c0-06df-43be-80ce-60586dd461c9/sequence-graph-15dee/0.log" Oct 30 22:58:44.937622 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:44.937586 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-4ace4-86496f9c9-lqblh_dd617f4a-e247-485e-85c7-83e715540f25/sequence-graph-4ace4/0.log" Oct 30 22:58:44.954207 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:44.954177 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" podUID="dd617f4a-e247-485e-85c7-83e715540f25" containerName="sequence-graph-4ace4" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:58:45.013302 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:45.013280 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-94a5a-5bcd4597f4-krr2t_241c8943-6710-48e5-b281-454b94b970be/sequence-graph-94a5a/0.log" Oct 30 22:58:46.327083 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:46.327030 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" podUID="490439c0-06df-43be-80ce-60586dd461c9" containerName="sequence-graph-15dee" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:58:46.507074 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:46.507033 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-15dee-557b5b7697-lmt2r_490439c0-06df-43be-80ce-60586dd461c9/sequence-graph-15dee/0.log" Oct 30 22:58:46.569771 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:46.569738 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-4ace4-86496f9c9-lqblh_dd617f4a-e247-485e-85c7-83e715540f25/sequence-graph-4ace4/0.log" Oct 30 22:58:46.637393 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:46.637367 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-94a5a-5bcd4597f4-krr2t_241c8943-6710-48e5-b281-454b94b970be/sequence-graph-94a5a/0.log" Oct 30 22:58:47.604488 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:47.604439 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" podUID="241c8943-6710-48e5-b281-454b94b970be" containerName="sequence-graph-94a5a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:58:48.098729 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:48.098699 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-15dee-557b5b7697-lmt2r_490439c0-06df-43be-80ce-60586dd461c9/sequence-graph-15dee/0.log" Oct 30 22:58:48.162811 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:48.162782 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-4ace4-86496f9c9-lqblh_dd617f4a-e247-485e-85c7-83e715540f25/sequence-graph-4ace4/0.log" Oct 30 22:58:48.226677 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:48.226649 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-94a5a-5bcd4597f4-krr2t_241c8943-6710-48e5-b281-454b94b970be/sequence-graph-94a5a/0.log" Oct 30 22:58:49.646867 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:49.646834 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-15dee-557b5b7697-lmt2r_490439c0-06df-43be-80ce-60586dd461c9/sequence-graph-15dee/0.log" Oct 30 22:58:49.710262 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:49.710238 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-4ace4-86496f9c9-lqblh_dd617f4a-e247-485e-85c7-83e715540f25/sequence-graph-4ace4/0.log" Oct 30 22:58:49.773434 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:49.773403 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-94a5a-5bcd4597f4-krr2t_241c8943-6710-48e5-b281-454b94b970be/sequence-graph-94a5a/0.log" Oct 30 22:58:49.953746 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:49.953659 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" podUID="dd617f4a-e247-485e-85c7-83e715540f25" containerName="sequence-graph-4ace4" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:58:51.179538 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:51.179507 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-15dee-557b5b7697-lmt2r_490439c0-06df-43be-80ce-60586dd461c9/sequence-graph-15dee/0.log" Oct 30 22:58:51.247946 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:51.247914 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-4ace4-86496f9c9-lqblh_dd617f4a-e247-485e-85c7-83e715540f25/sequence-graph-4ace4/0.log" Oct 30 22:58:51.315856 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:51.315827 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-94a5a-5bcd4597f4-krr2t_241c8943-6710-48e5-b281-454b94b970be/sequence-graph-94a5a/0.log" Oct 30 22:58:51.326413 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:51.326378 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" podUID="490439c0-06df-43be-80ce-60586dd461c9" containerName="sequence-graph-15dee" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:58:52.605204 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:52.605165 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" podUID="241c8943-6710-48e5-b281-454b94b970be" containerName="sequence-graph-94a5a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:58:52.697178 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:52.697151 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-15dee-557b5b7697-lmt2r_490439c0-06df-43be-80ce-60586dd461c9/sequence-graph-15dee/0.log" Oct 30 22:58:52.765499 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:52.765470 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-4ace4-86496f9c9-lqblh_dd617f4a-e247-485e-85c7-83e715540f25/sequence-graph-4ace4/0.log" Oct 30 22:58:52.832327 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:52.832300 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-94a5a-5bcd4597f4-krr2t_241c8943-6710-48e5-b281-454b94b970be/sequence-graph-94a5a/0.log" Oct 30 22:58:53.998534 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:53.998501 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:58:54.235648 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:54.235620 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-15dee-557b5b7697-lmt2r_490439c0-06df-43be-80ce-60586dd461c9/sequence-graph-15dee/0.log" Oct 30 22:58:54.298418 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:54.298345 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-4ace4-86496f9c9-lqblh_dd617f4a-e247-485e-85c7-83e715540f25/sequence-graph-4ace4/0.log" Oct 30 22:58:54.361793 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:54.361765 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-94a5a-5bcd4597f4-krr2t_241c8943-6710-48e5-b281-454b94b970be/sequence-graph-94a5a/0.log" Oct 30 22:58:54.954152 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:54.954107 4615 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" podUID="dd617f4a-e247-485e-85c7-83e715540f25" containerName="sequence-graph-4ace4" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 22:58:55.735027 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:55.734998 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-15dee-557b5b7697-lmt2r_490439c0-06df-43be-80ce-60586dd461c9/sequence-graph-15dee/0.log" Oct 30 22:58:55.793209 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:55.793175 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-4ace4-86496f9c9-lqblh_dd617f4a-e247-485e-85c7-83e715540f25/sequence-graph-4ace4/0.log" Oct 30 22:58:55.855545 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:55.853079 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw0-1b4518b221_sequence-graph-94a5a-5bcd4597f4-krr2t_241c8943-6710-48e5-b281-454b94b970be/sequence-graph-94a5a/0.log" Oct 30 22:58:55.863344 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:55.863314 4615 generic.go:356] "Generic (PLEG): container finished" podID="241c8943-6710-48e5-b281-454b94b970be" containerID="cf2ebea75ca962372265657be43ee22349e25b04e4992fd628487c637433a4f8" exitCode=0 Oct 30 22:58:55.863469 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:55.863418 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" event={"ID":"241c8943-6710-48e5-b281-454b94b970be","Type":"ContainerDied","Data":"cf2ebea75ca962372265657be43ee22349e25b04e4992fd628487c637433a4f8"} Oct 30 22:58:55.865470 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:55.865444 4615 generic.go:356] "Generic (PLEG): container finished" podID="dd617f4a-e247-485e-85c7-83e715540f25" containerID="b8667b84ee8a2b5848996e446102f03d58392555d723f28dd53843774a60eec9" exitCode=0 Oct 30 22:58:55.865600 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:55.865560 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" event={"ID":"dd617f4a-e247-485e-85c7-83e715540f25","Type":"ContainerDied","Data":"b8667b84ee8a2b5848996e446102f03d58392555d723f28dd53843774a60eec9"} Oct 30 22:58:55.867301 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:55.867267 4615 generic.go:356] "Generic (PLEG): container finished" podID="490439c0-06df-43be-80ce-60586dd461c9" containerID="efc34ce3d208a8eb2e342483364956a1acc3483e07df0de2911ac96b8f788f88" exitCode=137 Oct 30 22:58:55.867432 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:55.867331 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" event={"ID":"490439c0-06df-43be-80ce-60586dd461c9","Type":"ContainerDied","Data":"efc34ce3d208a8eb2e342483364956a1acc3483e07df0de2911ac96b8f788f88"} Oct 30 22:58:55.984425 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:55.984402 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" Oct 30 22:58:56.025812 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.025572 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" Oct 30 22:58:56.030392 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.030369 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" Oct 30 22:58:56.114628 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.114594 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/490439c0-06df-43be-80ce-60586dd461c9-openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490439c0-06df-43be-80ce-60586dd461c9-openshift-service-ca-bundle\") pod \"490439c0-06df-43be-80ce-60586dd461c9\" (UID: \"490439c0-06df-43be-80ce-60586dd461c9\") " Oct 30 22:58:56.114815 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.114657 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dd617f4a-e247-485e-85c7-83e715540f25-proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd617f4a-e247-485e-85c7-83e715540f25-proxy-tls\") pod \"dd617f4a-e247-485e-85c7-83e715540f25\" (UID: \"dd617f4a-e247-485e-85c7-83e715540f25\") " Oct 30 22:58:56.114815 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.114683 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/241c8943-6710-48e5-b281-454b94b970be-proxy-tls\" (UniqueName: \"kubernetes.io/secret/241c8943-6710-48e5-b281-454b94b970be-proxy-tls\") pod \"241c8943-6710-48e5-b281-454b94b970be\" (UID: \"241c8943-6710-48e5-b281-454b94b970be\") " Oct 30 22:58:56.114815 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.114701 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dd617f4a-e247-485e-85c7-83e715540f25-openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd617f4a-e247-485e-85c7-83e715540f25-openshift-service-ca-bundle\") pod \"dd617f4a-e247-485e-85c7-83e715540f25\" (UID: \"dd617f4a-e247-485e-85c7-83e715540f25\") " Oct 30 22:58:56.114815 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.114741 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/490439c0-06df-43be-80ce-60586dd461c9-proxy-tls\" (UniqueName: \"kubernetes.io/secret/490439c0-06df-43be-80ce-60586dd461c9-proxy-tls\") pod \"490439c0-06df-43be-80ce-60586dd461c9\" (UID: \"490439c0-06df-43be-80ce-60586dd461c9\") " Oct 30 22:58:56.115060 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.114884 4615 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/241c8943-6710-48e5-b281-454b94b970be-openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241c8943-6710-48e5-b281-454b94b970be-openshift-service-ca-bundle\") pod \"241c8943-6710-48e5-b281-454b94b970be\" (UID: \"241c8943-6710-48e5-b281-454b94b970be\") " Oct 30 22:58:56.115124 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.115075 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/490439c0-06df-43be-80ce-60586dd461c9-openshift-service-ca-bundle" pod "490439c0-06df-43be-80ce-60586dd461c9" (UID: "490439c0-06df-43be-80ce-60586dd461c9"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:58:56.115372 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.115136 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd617f4a-e247-485e-85c7-83e715540f25-openshift-service-ca-bundle" pod "dd617f4a-e247-485e-85c7-83e715540f25" (UID: "dd617f4a-e247-485e-85c7-83e715540f25"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:58:56.115372 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.115238 4615 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/490439c0-06df-43be-80ce-60586dd461c9-openshift-service-ca-bundle\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:58:56.115372 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.115260 4615 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd617f4a-e247-485e-85c7-83e715540f25-openshift-service-ca-bundle\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:58:56.115677 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.115654 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/241c8943-6710-48e5-b281-454b94b970be-openshift-service-ca-bundle" pod "241c8943-6710-48e5-b281-454b94b970be" (UID: "241c8943-6710-48e5-b281-454b94b970be"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 22:58:56.116925 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.116883 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/490439c0-06df-43be-80ce-60586dd461c9-proxy-tls" pod "490439c0-06df-43be-80ce-60586dd461c9" (UID: "490439c0-06df-43be-80ce-60586dd461c9"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:58:56.116988 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.116921 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd617f4a-e247-485e-85c7-83e715540f25-proxy-tls" pod "dd617f4a-e247-485e-85c7-83e715540f25" (UID: "dd617f4a-e247-485e-85c7-83e715540f25"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:58:56.117207 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.117194 4615 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/241c8943-6710-48e5-b281-454b94b970be-proxy-tls" pod "241c8943-6710-48e5-b281-454b94b970be" (UID: "241c8943-6710-48e5-b281-454b94b970be"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 22:58:56.215757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.215707 4615 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/490439c0-06df-43be-80ce-60586dd461c9-proxy-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:58:56.215757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.215750 4615 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/241c8943-6710-48e5-b281-454b94b970be-openshift-service-ca-bundle\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:58:56.215757 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.215762 4615 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dd617f4a-e247-485e-85c7-83e715540f25-proxy-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:58:56.215974 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.215773 4615 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/241c8943-6710-48e5-b281-454b94b970be-proxy-tls\") on node \"ip-10-0-131-94.ec2.internal\" DevicePath \"\"" Oct 30 22:58:56.872087 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.872054 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" Oct 30 22:58:56.872534 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.872083 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r" event={"ID":"490439c0-06df-43be-80ce-60586dd461c9","Type":"ContainerDied","Data":"85c11b1b92531e1cf54ccc6c035b19ed177ba17cd301c006db51af6087444142"} Oct 30 22:58:56.872534 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.872152 4615 scope.go:122] "RemoveContainer" containerID="efc34ce3d208a8eb2e342483364956a1acc3483e07df0de2911ac96b8f788f88" Oct 30 22:58:56.873418 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.873345 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" Oct 30 22:58:56.873418 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.873365 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t" event={"ID":"241c8943-6710-48e5-b281-454b94b970be","Type":"ContainerDied","Data":"72b821d29eacb934408c91dfe6d2b9f3d7ee1a2b5205e204039fa967702c80fc"} Oct 30 22:58:56.874624 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.874601 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" event={"ID":"dd617f4a-e247-485e-85c7-83e715540f25","Type":"ContainerDied","Data":"3f599bddfd0d5cf2e2e212312a29643755f33d8609963dfe84d9c25b409bdfd0"} Oct 30 22:58:56.874624 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.874618 4615 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh" Oct 30 22:58:56.882682 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.882551 4615 scope.go:122] "RemoveContainer" containerID="cf2ebea75ca962372265657be43ee22349e25b04e4992fd628487c637433a4f8" Oct 30 22:58:56.889988 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.889970 4615 scope.go:122] "RemoveContainer" containerID="b8667b84ee8a2b5848996e446102f03d58392555d723f28dd53843774a60eec9" Oct 30 22:58:56.926342 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.926321 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh"] Oct 30 22:58:56.937182 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.937163 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-1b4518b221/sequence-graph-4ace4-86496f9c9-lqblh"] Oct 30 22:58:56.972086 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.972057 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r"] Oct 30 22:58:56.976699 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:56.976679 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-1b4518b221/sequence-graph-15dee-557b5b7697-lmt2r"] Oct 30 22:58:57.011590 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:57.011566 4615 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t"] Oct 30 22:58:57.016310 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:57.016289 4615 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-1b4518b221/sequence-graph-94a5a-5bcd4597f4-krr2t"] Oct 30 22:58:57.232493 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:57.232465 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="241c8943-6710-48e5-b281-454b94b970be" path="/var/lib/kubelet/pods/241c8943-6710-48e5-b281-454b94b970be/volumes" Oct 30 22:58:57.232830 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:57.232816 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="490439c0-06df-43be-80ce-60586dd461c9" path="/var/lib/kubelet/pods/490439c0-06df-43be-80ce-60586dd461c9/volumes" Oct 30 22:58:57.233152 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:58:57.233138 4615 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd617f4a-e247-485e-85c7-83e715540f25" path="/var/lib/kubelet/pods/dd617f4a-e247-485e-85c7-83e715540f25/volumes" Oct 30 22:58:59.762306 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:58:59.762265 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:59:03.029027 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:03.028991 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:59:04.861933 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:04.861904 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:59:05.014479 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:05.014447 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:59:05.913648 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:05.913614 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:59:11.874508 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:11.874469 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:59:12.004420 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:12.004382 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:59:27.915206 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:27.915175 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:59:28.130202 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130163 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9m9rt/must-gather-4fwwz"] Oct 30 22:59:28.130492 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130479 4615 cpu_manager.go:414] "containerMap: removing container" podUID="876dbfb5-875f-4336-8247-982a6ac58d37" containerName="ensemble-graph-83e67" Oct 30 22:59:28.130534 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130494 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="876dbfb5-875f-4336-8247-982a6ac58d37" containerName="ensemble-graph-83e67" Oct 30 22:59:28.130534 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130510 4615 cpu_manager.go:414] "containerMap: removing container" podUID="dd617f4a-e247-485e-85c7-83e715540f25" containerName="sequence-graph-4ace4" Oct 30 22:59:28.130534 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130519 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd617f4a-e247-485e-85c7-83e715540f25" containerName="sequence-graph-4ace4" Oct 30 22:59:28.130534 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130528 4615 cpu_manager.go:414] "containerMap: removing container" podUID="490439c0-06df-43be-80ce-60586dd461c9" containerName="sequence-graph-15dee" Oct 30 22:59:28.130534 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130533 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="490439c0-06df-43be-80ce-60586dd461c9" containerName="sequence-graph-15dee" Oct 30 22:59:28.130680 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130540 4615 cpu_manager.go:414] "containerMap: removing container" podUID="241c8943-6710-48e5-b281-454b94b970be" containerName="sequence-graph-94a5a" Oct 30 22:59:28.130680 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130545 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="241c8943-6710-48e5-b281-454b94b970be" containerName="sequence-graph-94a5a" Oct 30 22:59:28.130680 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130550 4615 cpu_manager.go:414] "containerMap: removing container" podUID="9b7c3e3f-a09b-4eaf-91a9-f7809e028efd" containerName="kserve-container" Oct 30 22:59:28.130680 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130554 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9b7c3e3f-a09b-4eaf-91a9-f7809e028efd" containerName="kserve-container" Oct 30 22:59:28.130680 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130567 4615 cpu_manager.go:414] "containerMap: removing container" podUID="1bf494db-f7a0-4126-96aa-3819315dc2a0" containerName="ensemble-graph-c1ed1" Oct 30 22:59:28.130680 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130571 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1bf494db-f7a0-4126-96aa-3819315dc2a0" containerName="ensemble-graph-c1ed1" Oct 30 22:59:28.130680 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130581 4615 cpu_manager.go:414] "containerMap: removing container" podUID="a58e4278-3be4-415f-90f2-6a3c42afbb6f" containerName="splitter-graph-c00ac" Oct 30 22:59:28.130680 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130588 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a58e4278-3be4-415f-90f2-6a3c42afbb6f" containerName="splitter-graph-c00ac" Oct 30 22:59:28.130680 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130598 4615 cpu_manager.go:414] "containerMap: removing container" podUID="b64d9a50-7346-40dc-973b-5949ae6b977c" containerName="kube-rbac-proxy" Oct 30 22:59:28.130680 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130602 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b64d9a50-7346-40dc-973b-5949ae6b977c" containerName="kube-rbac-proxy" Oct 30 22:59:28.130680 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130613 4615 cpu_manager.go:414] "containerMap: removing container" podUID="9b7c3e3f-a09b-4eaf-91a9-f7809e028efd" containerName="kube-rbac-proxy" Oct 30 22:59:28.130680 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130617 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9b7c3e3f-a09b-4eaf-91a9-f7809e028efd" containerName="kube-rbac-proxy" Oct 30 22:59:28.130680 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130622 4615 cpu_manager.go:414] "containerMap: removing container" podUID="b64d9a50-7346-40dc-973b-5949ae6b977c" containerName="kserve-container" Oct 30 22:59:28.130680 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130626 4615 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b64d9a50-7346-40dc-973b-5949ae6b977c" containerName="kserve-container" Oct 30 22:59:28.130680 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130682 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="876dbfb5-875f-4336-8247-982a6ac58d37" containerName="ensemble-graph-83e67" Oct 30 22:59:28.131127 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130690 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd617f4a-e247-485e-85c7-83e715540f25" containerName="sequence-graph-4ace4" Oct 30 22:59:28.131127 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130696 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="490439c0-06df-43be-80ce-60586dd461c9" containerName="sequence-graph-15dee" Oct 30 22:59:28.131127 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130702 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="241c8943-6710-48e5-b281-454b94b970be" containerName="sequence-graph-94a5a" Oct 30 22:59:28.131127 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130707 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="9b7c3e3f-a09b-4eaf-91a9-f7809e028efd" containerName="kserve-container" Oct 30 22:59:28.131127 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130714 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="1bf494db-f7a0-4126-96aa-3819315dc2a0" containerName="ensemble-graph-c1ed1" Oct 30 22:59:28.131127 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130720 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="a58e4278-3be4-415f-90f2-6a3c42afbb6f" containerName="splitter-graph-c00ac" Oct 30 22:59:28.131127 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130725 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="b64d9a50-7346-40dc-973b-5949ae6b977c" containerName="kube-rbac-proxy" Oct 30 22:59:28.131127 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130730 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="9b7c3e3f-a09b-4eaf-91a9-f7809e028efd" containerName="kube-rbac-proxy" Oct 30 22:59:28.131127 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.130734 4615 memory_manager.go:365] "RemoveStaleState removing state" podUID="b64d9a50-7346-40dc-973b-5949ae6b977c" containerName="kserve-container" Oct 30 22:59:28.133837 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.133817 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9m9rt/must-gather-4fwwz" Oct 30 22:59:28.150114 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.150090 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9m9rt\"/\"kube-root-ca.crt\"" Oct 30 22:59:28.161015 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.160989 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcpbb\" (UniqueName: \"kubernetes.io/projected/774b99fc-2472-45c5-9575-8d12c7c63003-kube-api-access-xcpbb\") pod \"must-gather-4fwwz\" (UID: \"774b99fc-2472-45c5-9575-8d12c7c63003\") " pod="openshift-must-gather-9m9rt/must-gather-4fwwz" Oct 30 22:59:28.161131 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.161029 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/774b99fc-2472-45c5-9575-8d12c7c63003-must-gather-output\") pod \"must-gather-4fwwz\" (UID: \"774b99fc-2472-45c5-9575-8d12c7c63003\") " pod="openshift-must-gather-9m9rt/must-gather-4fwwz" Oct 30 22:59:28.162186 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.162165 4615 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9m9rt\"/\"default-dockercfg-q8f29\"" Oct 30 22:59:28.163154 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.163138 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9m9rt/must-gather-4fwwz"] Oct 30 22:59:28.171903 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.171858 4615 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9m9rt\"/\"openshift-service-ca.crt\"" Oct 30 22:59:28.184397 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.184374 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:59:28.262187 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.262166 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/774b99fc-2472-45c5-9575-8d12c7c63003-must-gather-output\") pod \"must-gather-4fwwz\" (UID: \"774b99fc-2472-45c5-9575-8d12c7c63003\") " pod="openshift-must-gather-9m9rt/must-gather-4fwwz" Oct 30 22:59:28.262277 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.262250 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcpbb\" (UniqueName: \"kubernetes.io/projected/774b99fc-2472-45c5-9575-8d12c7c63003-kube-api-access-xcpbb\") pod \"must-gather-4fwwz\" (UID: \"774b99fc-2472-45c5-9575-8d12c7c63003\") " pod="openshift-must-gather-9m9rt/must-gather-4fwwz" Oct 30 22:59:28.262538 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.262519 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/774b99fc-2472-45c5-9575-8d12c7c63003-must-gather-output\") pod \"must-gather-4fwwz\" (UID: \"774b99fc-2472-45c5-9575-8d12c7c63003\") " pod="openshift-must-gather-9m9rt/must-gather-4fwwz" Oct 30 22:59:28.281645 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.281610 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcpbb\" (UniqueName: \"kubernetes.io/projected/774b99fc-2472-45c5-9575-8d12c7c63003-kube-api-access-xcpbb\") pod \"must-gather-4fwwz\" (UID: \"774b99fc-2472-45c5-9575-8d12c7c63003\") " pod="openshift-must-gather-9m9rt/must-gather-4fwwz" Oct 30 22:59:28.335794 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.335772 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:59:28.456205 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.456146 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9m9rt/must-gather-4fwwz" Oct 30 22:59:28.743326 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.743301 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9m9rt/must-gather-4fwwz"] Oct 30 22:59:28.744719 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:59:28.744693 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod774b99fc_2472_45c5_9575_8d12c7c63003.slice/crio-869ce3f912789f4595f89c9728307c27532216ba55f81a012a09bf4728293e85 WatchSource:0}: Error finding container 869ce3f912789f4595f89c9728307c27532216ba55f81a012a09bf4728293e85: Status 404 returned error can't find the container with id 869ce3f912789f4595f89c9728307c27532216ba55f81a012a09bf4728293e85 Oct 30 22:59:28.746621 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.746601 4615 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 22:59:28.997094 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:28.997013 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9m9rt/must-gather-4fwwz" event={"ID":"774b99fc-2472-45c5-9575-8d12c7c63003","Type":"ContainerStarted","Data":"869ce3f912789f4595f89c9728307c27532216ba55f81a012a09bf4728293e85"} Oct 30 22:59:29.767594 ip-10-0-131-94 kubenswrapper[4615]: E1030 22:59:29.767513 4615 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-gs6ts" containerName="extractor" Oct 30 22:59:30.003349 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:30.003314 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9m9rt/must-gather-4fwwz" event={"ID":"774b99fc-2472-45c5-9575-8d12c7c63003","Type":"ContainerStarted","Data":"48252001b7835de4c65236cc59be3b84bc530775395e6760d61f5dba9dadbc6b"} Oct 30 22:59:30.003744 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:30.003364 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9m9rt/must-gather-4fwwz" event={"ID":"774b99fc-2472-45c5-9575-8d12c7c63003","Type":"ContainerStarted","Data":"f677c9ad87cbed354150f08edf75cf35b7bb0df9e73b19011e78ffe9c05b2daa"} Oct 30 22:59:30.043399 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:30.043303 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-9m9rt/must-gather-4fwwz" podStartSLOduration=1.32921315 podStartE2EDuration="2.043287495s" totalImagesPullingTime="714.074345ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:59:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:59:30.041721669 +0000 UTC m=+3086.957407971" watchObservedRunningTime="2026-10-30 22:59:30.043287495 +0000 UTC m=+3086.958973823" Oct 30 22:59:31.990036 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:31.990005 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gsvm9_bdc1c8ae-38f2-4bb3-be0e-1920a21ee0e9/global-pull-secret-syncer/0.log" Oct 30 22:59:32.621689 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:32.621656 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" Oct 30 22:59:33.134898 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:33.134869 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-94.ec2.internal_f39ce36b79a5ed24b44c91f0e6e003f7/haproxy/0.log" Oct 30 22:59:33.731020 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:33.730979 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:59:33.890066 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:33.882976 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:59:38.166455 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:38.166413 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3191aa1c-021d-47b1-82ac-0edd9e17325b/alertmanager/0.log" Oct 30 22:59:38.286372 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:38.286345 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3191aa1c-021d-47b1-82ac-0edd9e17325b/config-reloader/0.log" Oct 30 22:59:38.403342 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:38.403302 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3191aa1c-021d-47b1-82ac-0edd9e17325b/kube-rbac-proxy-web/0.log" Oct 30 22:59:38.521078 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:38.520987 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3191aa1c-021d-47b1-82ac-0edd9e17325b/kube-rbac-proxy/0.log" Oct 30 22:59:38.637833 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:38.637805 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3191aa1c-021d-47b1-82ac-0edd9e17325b/kube-rbac-proxy-metric/0.log" Oct 30 22:59:38.752785 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:38.752754 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3191aa1c-021d-47b1-82ac-0edd9e17325b/prom-label-proxy/0.log" Oct 30 22:59:38.880241 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:38.880216 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3191aa1c-021d-47b1-82ac-0edd9e17325b/init-config-reloader/0.log" Oct 30 22:59:39.143001 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:39.142905 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-v4894_b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6/kube-state-metrics/0.log" Oct 30 22:59:39.258198 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:39.258171 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-v4894_b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6/kube-rbac-proxy-main/0.log" Oct 30 22:59:39.374597 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:39.374569 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-v4894_b32f8fd6-630e-46b6-98c8-cc2ac7c52cb6/kube-rbac-proxy-self/0.log" Oct 30 22:59:39.491815 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:39.491748 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-64c486767d-86tlj_1fcba11f-af23-40ad-8f18-f3a14c48d1d5/metrics-server/0.log" Oct 30 22:59:39.738596 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:39.738559 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fzbfr_829e5887-6e47-4940-8c75-8d07907ab739/node-exporter/0.log" Oct 30 22:59:39.854325 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:39.854207 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fzbfr_829e5887-6e47-4940-8c75-8d07907ab739/kube-rbac-proxy/0.log" Oct 30 22:59:39.968948 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:39.968916 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fzbfr_829e5887-6e47-4940-8c75-8d07907ab739/init-textfile/0.log" Oct 30 22:59:41.176992 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:41.176960 4615 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc"] Oct 30 22:59:41.182519 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:41.182494 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc" Oct 30 22:59:41.205522 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:41.205493 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc"] Oct 30 22:59:41.256693 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:41.256659 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/63e5b1c9-4d1d-4041-8f5a-fa1e3e360534-podres\") pod \"perf-node-gather-daemonset-dlmnc\" (UID: \"63e5b1c9-4d1d-4041-8f5a-fa1e3e360534\") " pod="openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc" Oct 30 22:59:41.256693 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:41.256693 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6tmv\" (UniqueName: \"kubernetes.io/projected/63e5b1c9-4d1d-4041-8f5a-fa1e3e360534-kube-api-access-g6tmv\") pod \"perf-node-gather-daemonset-dlmnc\" (UID: \"63e5b1c9-4d1d-4041-8f5a-fa1e3e360534\") " pod="openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc" Oct 30 22:59:41.256962 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:41.256715 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63e5b1c9-4d1d-4041-8f5a-fa1e3e360534-sys\") pod \"perf-node-gather-daemonset-dlmnc\" (UID: \"63e5b1c9-4d1d-4041-8f5a-fa1e3e360534\") " pod="openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc" Oct 30 22:59:41.256962 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:41.256733 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/63e5b1c9-4d1d-4041-8f5a-fa1e3e360534-proc\") pod \"perf-node-gather-daemonset-dlmnc\" (UID: \"63e5b1c9-4d1d-4041-8f5a-fa1e3e360534\") " pod="openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc" Oct 30 22:59:41.256962 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:41.256873 4615 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63e5b1c9-4d1d-4041-8f5a-fa1e3e360534-lib-modules\") pod \"perf-node-gather-daemonset-dlmnc\" (UID: \"63e5b1c9-4d1d-4041-8f5a-fa1e3e360534\") " pod="openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc" Oct 30 22:59:41.358027 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:41.357989 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/63e5b1c9-4d1d-4041-8f5a-fa1e3e360534-proc\") pod \"perf-node-gather-daemonset-dlmnc\" (UID: \"63e5b1c9-4d1d-4041-8f5a-fa1e3e360534\") " pod="openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc" Oct 30 22:59:41.358224 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:41.358037 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63e5b1c9-4d1d-4041-8f5a-fa1e3e360534-lib-modules\") pod \"perf-node-gather-daemonset-dlmnc\" (UID: \"63e5b1c9-4d1d-4041-8f5a-fa1e3e360534\") " pod="openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc" Oct 30 22:59:41.358224 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:41.358127 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/63e5b1c9-4d1d-4041-8f5a-fa1e3e360534-proc\") pod \"perf-node-gather-daemonset-dlmnc\" (UID: \"63e5b1c9-4d1d-4041-8f5a-fa1e3e360534\") " pod="openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc" Oct 30 22:59:41.358224 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:41.358197 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/63e5b1c9-4d1d-4041-8f5a-fa1e3e360534-podres\") pod \"perf-node-gather-daemonset-dlmnc\" (UID: \"63e5b1c9-4d1d-4041-8f5a-fa1e3e360534\") " pod="openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc" Oct 30 22:59:41.358224 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:41.358219 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6tmv\" (UniqueName: \"kubernetes.io/projected/63e5b1c9-4d1d-4041-8f5a-fa1e3e360534-kube-api-access-g6tmv\") pod \"perf-node-gather-daemonset-dlmnc\" (UID: \"63e5b1c9-4d1d-4041-8f5a-fa1e3e360534\") " pod="openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc" Oct 30 22:59:41.358399 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:41.358228 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63e5b1c9-4d1d-4041-8f5a-fa1e3e360534-lib-modules\") pod \"perf-node-gather-daemonset-dlmnc\" (UID: \"63e5b1c9-4d1d-4041-8f5a-fa1e3e360534\") " pod="openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc" Oct 30 22:59:41.358399 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:41.358240 4615 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63e5b1c9-4d1d-4041-8f5a-fa1e3e360534-sys\") pod \"perf-node-gather-daemonset-dlmnc\" (UID: \"63e5b1c9-4d1d-4041-8f5a-fa1e3e360534\") " pod="openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc" Oct 30 22:59:41.358399 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:41.358268 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63e5b1c9-4d1d-4041-8f5a-fa1e3e360534-sys\") pod \"perf-node-gather-daemonset-dlmnc\" (UID: \"63e5b1c9-4d1d-4041-8f5a-fa1e3e360534\") " pod="openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc" Oct 30 22:59:41.358399 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:41.358278 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/63e5b1c9-4d1d-4041-8f5a-fa1e3e360534-podres\") pod \"perf-node-gather-daemonset-dlmnc\" (UID: \"63e5b1c9-4d1d-4041-8f5a-fa1e3e360534\") " pod="openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc" Oct 30 22:59:41.377371 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:41.377338 4615 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6tmv\" (UniqueName: \"kubernetes.io/projected/63e5b1c9-4d1d-4041-8f5a-fa1e3e360534-kube-api-access-g6tmv\") pod \"perf-node-gather-daemonset-dlmnc\" (UID: \"63e5b1c9-4d1d-4041-8f5a-fa1e3e360534\") " pod="openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc" Oct 30 22:59:41.501235 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:41.501166 4615 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc" Oct 30 22:59:41.791122 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:41.790718 4615 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc"] Oct 30 22:59:41.792682 ip-10-0-131-94 kubenswrapper[4615]: W1030 22:59:41.792641 4615 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod63e5b1c9_4d1d_4041_8f5a_fa1e3e360534.slice/crio-34cac23fc09ca94aaeba5d830a6fc726f1ee5eed0e0f8a70174b02f0a7c35089 WatchSource:0}: Error finding container 34cac23fc09ca94aaeba5d830a6fc726f1ee5eed0e0f8a70174b02f0a7c35089: Status 404 returned error can't find the container with id 34cac23fc09ca94aaeba5d830a6fc726f1ee5eed0e0f8a70174b02f0a7c35089 Oct 30 22:59:42.023583 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:42.023554 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-69twj_93e3dae5-1990-4737-91ae-178d25c4ff1d/prometheus-operator/0.log" Oct 30 22:59:42.075581 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:42.075502 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc" event={"ID":"63e5b1c9-4d1d-4041-8f5a-fa1e3e360534","Type":"ContainerStarted","Data":"2611db4178e3de3e3d04238dca16ce35b34a19969960604f2df9ef09bedd50a5"} Oct 30 22:59:42.075581 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:42.075551 4615 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc" event={"ID":"63e5b1c9-4d1d-4041-8f5a-fa1e3e360534","Type":"ContainerStarted","Data":"34cac23fc09ca94aaeba5d830a6fc726f1ee5eed0e0f8a70174b02f0a7c35089"} Oct 30 22:59:42.075581 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:42.075582 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc" Oct 30 22:59:42.121730 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:42.121611 4615 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc" podStartSLOduration=1.121594334 podStartE2EDuration="1.121594334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:59:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:59:42.121534942 +0000 UTC m=+3099.037221256" watchObservedRunningTime="2026-10-30 22:59:42.121594334 +0000 UTC m=+3099.037280639" Oct 30 22:59:42.152804 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:42.152772 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-69twj_93e3dae5-1990-4737-91ae-178d25c4ff1d/kube-rbac-proxy/0.log" Oct 30 22:59:42.274548 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:42.274520 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-v9mj4_9ba0eb49-d6b6-4b0b-b86c-97245a119b2c/prometheus-operator-admission-webhook/0.log" Oct 30 22:59:46.502795 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:46.502711 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-54cf75fc99-cf9gr_ec244118-01e7-4088-9048-e8004969bf96/console/0.log" Oct 30 22:59:46.626767 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:46.626733 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6dd789c8b7-cpcvr_12607685-74bf-471b-9c1d-ae94e6e9c6c7/download-server/0.log" Oct 30 22:59:48.089897 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:48.089855 4615 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9m9rt/perf-node-gather-daemonset-dlmnc" Oct 30 22:59:49.882871 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:49.882839 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xtbrk_3bf28a7e-a8ce-4f33-923a-5d1539978790/dns/0.log" Oct 30 22:59:49.993695 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:49.993665 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xtbrk_3bf28a7e-a8ce-4f33-923a-5d1539978790/kube-rbac-proxy/0.log" Oct 30 22:59:50.225366 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:50.225291 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q5gw7_c135a307-61bc-4e34-9b5c-b97793255db7/dns-node-resolver/0.log" Oct 30 22:59:51.749237 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:51.749204 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-h9qqf_9432e919-3bed-47a1-8b4e-4b979ebdb046/node-ca/0.log" Oct 30 22:59:54.077602 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:54.077554 4615 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-6qz5h_12b2045b-a302-47a1-b039-b8eadcc50c61/konnectivity-agent/0.log" limit=1024 Oct 30 22:59:54.572461 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:54.572430 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-khpnv_24e7e5b3-a4ea-4322-915b-5694ea1f3112/serve-healthcheck-canary/0.log" Oct 30 22:59:56.247790 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:56.247757 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gs6ts_780de4be-6a4c-4afa-9fee-a3391101d004/kube-rbac-proxy/0.log" Oct 30 22:59:56.360507 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:56.360473 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gs6ts_780de4be-6a4c-4afa-9fee-a3391101d004/exporter/0.log" Oct 30 22:59:56.474098 ip-10-0-131-94 kubenswrapper[4615]: I1030 22:59:56.474034 4615 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gs6ts_780de4be-6a4c-4afa-9fee-a3391101d004/extractor/0.log"