Oct 30 18:34:58.772611 ip-10-0-132-74 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Oct 30 18:34:58.772625 ip-10-0-132-74 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Oct 30 18:34:58.772634 ip-10-0-132-74 systemd[1]: kubelet.service: Failed with result 'resources'. Oct 30 18:34:58.772951 ip-10-0-132-74 systemd[1]: Failed to start Kubernetes Kubelet. Oct 30 18:35:09.015478 ip-10-0-132-74 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Oct 30 18:35:09.015495 ip-10-0-132-74 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot a546173e5f714725bb315f4119027579 -- Oct 30 18:36:54.284808 ip-10-0-132-74 systemd[1]: Starting Kubernetes Kubelet... Oct 30 18:36:54.737095 ip-10-0-132-74 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:36:54.737095 ip-10-0-132-74 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 18:36:54.737095 ip-10-0-132-74 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:36:54.737095 ip-10-0-132-74 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:36:54.740619 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740542 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:36:54.740619 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740618 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:36:54.740619 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740622 2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:36:54.740713 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740625 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:36:54.740713 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740629 2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:36:54.740713 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740631 2581 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:36:54.740713 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740634 2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:36:54.740713 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740637 2581 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:36:54.740713 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740640 2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:36:54.740713 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740643 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:36:54.740713 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740646 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:36:54.740713 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740648 2581 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:36:54.740713 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740651 2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:36:54.740713 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740654 2581 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:36:54.740713 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740656 2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:36:54.740713 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740659 2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:36:54.740713 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740662 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:36:54.740713 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740665 2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:36:54.740713 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740668 2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:36:54.740713 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740670 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:36:54.740713 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740674 2581 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:36:54.740713 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740677 2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:36:54.740713 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740679 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:36:54.741181 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740682 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:36:54.741181 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740684 2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:36:54.741181 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740687 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:36:54.741181 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740690 2581 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:36:54.741181 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740692 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:36:54.741181 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740695 2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:36:54.741181 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740698 2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:36:54.741181 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740701 2581 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:36:54.741181 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740703 2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:36:54.741181 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740706 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:36:54.741181 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740708 2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:36:54.741181 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740711 2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:36:54.741181 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740713 2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:36:54.741181 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740717 2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:36:54.741181 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740720 2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:36:54.741181 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740722 2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:36:54.741181 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740725 2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:36:54.741181 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740729 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:36:54.741181 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740731 2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:36:54.741181 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740734 2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:36:54.741671 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740737 2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:36:54.741671 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740739 2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:36:54.741671 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740744 2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:36:54.741671 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740749 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:36:54.741671 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740752 2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:36:54.741671 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740755 2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:36:54.741671 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740759 2581 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:36:54.741671 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740762 2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:36:54.741671 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740764 2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:36:54.741671 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740767 2581 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:36:54.741671 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740769 2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:36:54.741671 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740772 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:36:54.741671 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740774 2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:36:54.741671 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740777 2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:36:54.741671 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740780 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:36:54.741671 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740782 2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:36:54.741671 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740785 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:36:54.741671 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740787 2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:36:54.741671 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740790 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:36:54.741671 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740792 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:36:54.742173 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740795 2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:36:54.742173 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740797 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:36:54.742173 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740800 2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:36:54.742173 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740802 2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:36:54.742173 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740805 2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:36:54.742173 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740809 2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:36:54.742173 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740812 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:36:54.742173 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740815 2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:36:54.742173 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740817 2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:36:54.742173 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740820 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:36:54.742173 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740823 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:36:54.742173 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740826 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:36:54.742173 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740829 2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:36:54.742173 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740831 2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:36:54.742173 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740834 2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:36:54.742173 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740838 2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:36:54.742173 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740842 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:36:54.742173 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740846 2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:36:54.743275 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740849 2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:36:54.743275 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740858 2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:36:54.743275 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740863 2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:36:54.743275 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740866 2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:36:54.743275 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740869 2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:36:54.743275 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740872 2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:36:54.743275 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740875 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:36:54.743275 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740877 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:36:54.743275 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740880 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:36:54.743275 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740883 2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:36:54.743275 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740885 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:36:54.743275 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740888 2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:36:54.743275 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740891 2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:36:54.743275 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740893 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:36:54.743275 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740896 2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:36:54.743275 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740898 2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:36:54.743275 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740901 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:36:54.743275 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740903 2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:36:54.743275 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740906 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:36:54.743275 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740910 2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:36:54.743761 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740913 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:36:54.743761 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740915 2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:36:54.743761 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740918 2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:36:54.743761 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740921 2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:36:54.743761 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.740923 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:36:54.743761 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741732 2581 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:36:54.743761 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741741 2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:36:54.743761 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741746 2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:36:54.743761 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741749 2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:36:54.743761 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741752 2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:36:54.743761 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741756 2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:36:54.743761 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741760 2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:36:54.743761 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741763 2581 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:36:54.743761 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741766 2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:36:54.743761 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741768 2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:36:54.743761 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741771 2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:36:54.743761 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741773 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:36:54.743761 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741776 2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:36:54.743761 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741779 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:36:54.744216 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741781 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:36:54.744216 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741784 2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:36:54.744216 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741787 2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:36:54.744216 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741790 2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:36:54.744216 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741793 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:36:54.744216 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741795 2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:36:54.744216 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741797 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:36:54.744216 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741800 2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:36:54.744216 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741803 2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:36:54.744216 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741806 2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:36:54.744216 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741808 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:36:54.744216 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741811 2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:36:54.744216 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741814 2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:36:54.744216 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741817 2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:36:54.744216 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741819 2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:36:54.744216 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741822 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:36:54.744216 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741824 2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:36:54.744216 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741827 2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:36:54.744216 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741829 2581 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:36:54.744696 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741837 2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:36:54.744696 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741840 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:36:54.744696 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741843 2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:36:54.744696 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741846 2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:36:54.744696 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741848 2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:36:54.744696 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741851 2581 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:36:54.744696 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741853 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:36:54.744696 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741856 2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:36:54.744696 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741858 2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:36:54.744696 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741861 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:36:54.744696 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741863 2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:36:54.744696 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741866 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:36:54.744696 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741868 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:36:54.744696 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741871 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:36:54.744696 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741874 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:36:54.744696 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741877 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:36:54.744696 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741879 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:36:54.744696 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741882 2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:36:54.744696 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741885 2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:36:54.745168 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741887 2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:36:54.745168 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741890 2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:36:54.745168 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741892 2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:36:54.745168 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741895 2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:36:54.745168 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741897 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:36:54.745168 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741900 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:36:54.745168 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741902 2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:36:54.745168 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741905 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:36:54.745168 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741907 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:36:54.745168 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741910 2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:36:54.745168 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741912 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:36:54.745168 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741915 2581 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:36:54.745168 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741917 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:36:54.745168 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741920 2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:36:54.745168 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741928 2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:36:54.745168 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741930 2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:36:54.745168 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741933 2581 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:36:54.745168 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741935 2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:36:54.745168 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741938 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:36:54.745168 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741940 2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:36:54.745698 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741943 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:36:54.745698 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741945 2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:36:54.745698 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741948 2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:36:54.745698 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741950 2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:36:54.745698 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741953 2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:36:54.745698 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741955 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:36:54.745698 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741958 2581 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:36:54.745698 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741960 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:36:54.745698 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741964 2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:36:54.745698 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741966 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:36:54.745698 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741969 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:36:54.745698 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741972 2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:36:54.745698 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741974 2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:36:54.745698 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741977 2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:36:54.745698 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741979 2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:36:54.745698 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741983 2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:36:54.745698 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741985 2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:36:54.745698 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741988 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:36:54.745698 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741991 2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:36:54.745698 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741994 2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:36:54.746171 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741996 2581 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:36:54.746171 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.741999 2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:36:54.746171 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742001 2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:36:54.746171 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742004 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:36:54.746171 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742006 2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:36:54.746171 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742009 2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:36:54.746171 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742011 2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:36:54.746171 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742017 2581 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:36:54.746171 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742020 2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:36:54.746171 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742022 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:36:54.746171 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742025 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:36:54.746171 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742028 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:36:54.746171 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742031 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:36:54.746171 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742034 2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:36:54.746171 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742583 2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:36:54.746171 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742589 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:36:54.746171 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742593 2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:36:54.746171 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742596 2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:36:54.746171 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742599 2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:36:54.746171 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742602 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:36:54.746662 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742605 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:36:54.746662 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742608 2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:36:54.746662 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742610 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:36:54.746662 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742613 2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:36:54.746662 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742615 2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:36:54.746662 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742618 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:36:54.746662 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742620 2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:36:54.746662 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742623 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:36:54.746662 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742626 2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:36:54.746662 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742629 2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:36:54.746662 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742632 2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:36:54.746662 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742635 2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:36:54.746662 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742637 2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:36:54.746662 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742640 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:36:54.746662 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742643 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:36:54.746662 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742645 2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:36:54.746662 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742655 2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:36:54.746662 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742658 2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:36:54.746662 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742660 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:36:54.747105 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742668 2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:36:54.747105 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742671 2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:36:54.747105 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742674 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:36:54.747105 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742676 2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:36:54.747105 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742679 2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:36:54.747105 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742681 2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:36:54.747105 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742684 2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:36:54.747105 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742686 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:36:54.747105 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742689 2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:36:54.747105 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742692 2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:36:54.747105 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742695 2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:36:54.747105 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742697 2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:36:54.747105 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742700 2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:36:54.747105 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742703 2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:36:54.747105 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742705 2581 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:36:54.747105 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742708 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:36:54.747105 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742711 2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:36:54.747105 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742714 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:36:54.747105 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742716 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:36:54.747564 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742719 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:36:54.747564 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742722 2581 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:36:54.747564 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742724 2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:36:54.747564 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742727 2581 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:36:54.747564 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742729 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:36:54.747564 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742732 2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:36:54.747564 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742734 2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:36:54.747564 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742737 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:36:54.747564 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742739 2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:36:54.747564 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742743 2581 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:36:54.747564 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742745 2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:36:54.747564 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742748 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:36:54.747564 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742750 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:36:54.747564 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742753 2581 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:36:54.747564 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742761 2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:36:54.747564 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742764 2581 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:36:54.747564 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742766 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:36:54.747564 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742769 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:36:54.747564 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742771 2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:36:54.747564 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742774 2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:36:54.748041 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742777 2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:36:54.748041 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742780 2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:36:54.748041 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742782 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:36:54.748041 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742784 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:36:54.748041 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742787 2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:36:54.748041 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742790 2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:36:54.748041 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742793 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:36:54.748041 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742796 2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:36:54.748041 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742798 2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:36:54.748041 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742800 2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:36:54.748041 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742803 2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:36:54.748041 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742805 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:36:54.748041 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742808 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:36:54.748041 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742812 2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:36:54.748041 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742815 2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:36:54.748041 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742818 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:36:54.748041 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742821 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:36:54.748041 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742823 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:36:54.748041 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742826 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:36:54.748485 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742828 2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:36:54.748485 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742831 2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:36:54.748485 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742833 2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:36:54.748485 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742836 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:36:54.748485 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742840 2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:36:54.748485 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742843 2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:36:54.748485 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742845 2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:36:54.748485 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742848 2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:36:54.748485 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742856 2581 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:36:54.748485 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742859 2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:36:54.748485 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742862 2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:36:54.748485 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742864 2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:36:54.748485 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742867 2581 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:36:54.748485 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742870 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:36:54.748485 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742873 2581 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:36:54.748485 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742875 2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:36:54.748485 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742878 2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:36:54.748485 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742880 2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:36:54.748485 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742883 2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:36:54.748485 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742886 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742892 2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742895 2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.742897 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.743977 2581 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.743992 2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744001 2581 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744006 2581 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744011 2581 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744014 2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744019 2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744023 2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744026 2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744029 2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744033 2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744036 2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744039 2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744042 2581 flags.go:64] FLAG: --cgroup-root="" Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744048 2581 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744051 2581 flags.go:64] FLAG: --client-ca-file="" Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744054 2581 flags.go:64] FLAG: --cloud-provider="external" Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744057 2581 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744062 2581 flags.go:64] FLAG: --cluster-domain="" Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744065 2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 18:36:54.748975 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744068 2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744072 2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744075 2581 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744079 2581 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744083 2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744086 2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744089 2581 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744093 2581 flags.go:64] FLAG: --contention-profiling="false" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744096 2581 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744099 2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744103 2581 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744106 2581 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744110 2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744113 2581 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744116 2581 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744119 2581 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744122 2581 flags.go:64] FLAG: --enable-server="true" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744125 2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744130 2581 flags.go:64] FLAG: --event-burst="100" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744133 2581 flags.go:64] FLAG: --event-qps="50" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744136 2581 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744139 2581 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744142 2581 flags.go:64] FLAG: --eviction-hard="" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744147 2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 18:36:54.749558 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744150 2581 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744153 2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744158 2581 flags.go:64] FLAG: --eviction-soft="" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744161 2581 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744164 2581 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744167 2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744170 2581 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744174 2581 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744177 2581 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744180 2581 flags.go:64] FLAG: --feature-gates="" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744184 2581 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744187 2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744190 2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744194 2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744197 2581 flags.go:64] FLAG: --healthz-port="10248" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744200 2581 flags.go:64] FLAG: --help="false" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744203 2581 flags.go:64] FLAG: --hostname-override="ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744206 2581 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744210 2581 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744212 2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744216 2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744219 2581 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744222 2581 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 18:36:54.750159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744225 2581 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744228 2581 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744232 2581 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744234 2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744238 2581 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744241 2581 flags.go:64] FLAG: --kube-reserved="" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744244 2581 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744247 2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744250 2581 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744253 2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744255 2581 flags.go:64] FLAG: --lock-file="" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744259 2581 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744262 2581 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744266 2581 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744271 2581 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744274 2581 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744277 2581 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744281 2581 flags.go:64] FLAG: --logging-format="text" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744285 2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744288 2581 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744291 2581 flags.go:64] FLAG: --manifest-url="" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744294 2581 flags.go:64] FLAG: --manifest-url-header="" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744299 2581 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744302 2581 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744306 2581 flags.go:64] FLAG: --max-pods="110" Oct 30 18:36:54.750743 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744309 2581 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744312 2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744315 2581 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744318 2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744321 2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744324 2581 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744327 2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744334 2581 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744338 2581 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744341 2581 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744344 2581 flags.go:64] FLAG: --pod-cidr="" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744347 2581 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744350 2581 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744353 2581 flags.go:64] FLAG: --pods-per-core="0" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744357 2581 flags.go:64] FLAG: --port="10250" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744360 2581 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744363 2581 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-0fa259a1438c7dd7f" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744366 2581 flags.go:64] FLAG: --qos-reserved="" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744372 2581 flags.go:64] FLAG: --read-only-port="10255" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744375 2581 flags.go:64] FLAG: --register-node="true" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744378 2581 flags.go:64] FLAG: --register-with-taints="" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744382 2581 flags.go:64] FLAG: --registry-burst="10" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744385 2581 flags.go:64] FLAG: --registry-qps="5" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744388 2581 flags.go:64] FLAG: --reserved-cpus="" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744391 2581 flags.go:64] FLAG: --reserved-memory="" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744397 2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 18:36:54.751463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744401 2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744404 2581 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744407 2581 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744410 2581 flags.go:64] FLAG: --runonce="false" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744413 2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744416 2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744419 2581 flags.go:64] FLAG: --seccomp-default="false" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744423 2581 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744425 2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744428 2581 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744432 2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744435 2581 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744438 2581 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744440 2581 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744443 2581 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744446 2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744449 2581 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744453 2581 flags.go:64] FLAG: --system-cgroups="" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744456 2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744462 2581 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744465 2581 flags.go:64] FLAG: --tls-cert-file="" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744469 2581 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744473 2581 flags.go:64] FLAG: --tls-min-version="" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744476 2581 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744480 2581 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 18:36:54.752401 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744483 2581 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 18:36:54.753179 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744486 2581 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 18:36:54.753179 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744499 2581 flags.go:64] FLAG: --v="2" Oct 30 18:36:54.753179 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744504 2581 flags.go:64] FLAG: --version="false" Oct 30 18:36:54.753179 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744508 2581 flags.go:64] FLAG: --vmodule="" Oct 30 18:36:54.753179 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744513 2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 18:36:54.753179 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.744517 2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 18:36:54.753179 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744682 2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:36:54.753179 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744687 2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:36:54.753179 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744690 2581 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:36:54.753179 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744693 2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:36:54.753179 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744695 2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:36:54.753179 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744698 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:36:54.753179 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744701 2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:36:54.753179 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744704 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:36:54.753179 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744707 2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:36:54.753179 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744710 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:36:54.753179 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744712 2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:36:54.753179 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744715 2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:36:54.753179 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744718 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:36:54.753179 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744721 2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:36:54.753179 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744723 2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:36:54.753715 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744726 2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:36:54.753715 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744729 2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:36:54.753715 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744732 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:36:54.753715 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744734 2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:36:54.753715 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744737 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:36:54.753715 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744740 2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:36:54.753715 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744742 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:36:54.753715 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744745 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:36:54.753715 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744748 2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:36:54.753715 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744752 2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:36:54.753715 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744754 2581 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:36:54.753715 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744757 2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:36:54.753715 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744760 2581 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:36:54.753715 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744763 2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:36:54.753715 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744766 2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:36:54.753715 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744768 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:36:54.753715 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744772 2581 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:36:54.753715 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744775 2581 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:36:54.753715 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744779 2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:36:54.753715 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744782 2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:36:54.754183 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744784 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:36:54.754183 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744787 2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:36:54.754183 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744790 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:36:54.754183 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744794 2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:36:54.754183 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744798 2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:36:54.754183 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744801 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:36:54.754183 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744804 2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:36:54.754183 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744807 2581 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:36:54.754183 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744809 2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:36:54.754183 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744812 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:36:54.754183 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744814 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:36:54.754183 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744817 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:36:54.754183 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744820 2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:36:54.754183 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744822 2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:36:54.754183 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744825 2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:36:54.754183 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744828 2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:36:54.754183 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744830 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:36:54.754183 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744833 2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:36:54.754183 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744835 2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:36:54.754183 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744838 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:36:54.754687 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744841 2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:36:54.754687 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744844 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:36:54.754687 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744847 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:36:54.754687 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744850 2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:36:54.754687 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744859 2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:36:54.754687 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744863 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:36:54.754687 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744865 2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:36:54.754687 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744868 2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:36:54.754687 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744872 2581 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:36:54.754687 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744875 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:36:54.754687 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744878 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:36:54.754687 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744881 2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:36:54.754687 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744884 2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:36:54.754687 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744887 2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:36:54.754687 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744889 2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:36:54.754687 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744892 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:36:54.754687 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744894 2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:36:54.754687 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744898 2581 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:36:54.754687 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744901 2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:36:54.754687 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744903 2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:36:54.755185 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744905 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:36:54.755185 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744908 2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:36:54.755185 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744911 2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:36:54.755185 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744915 2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:36:54.755185 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744918 2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:36:54.755185 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744921 2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:36:54.755185 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744924 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:36:54.755185 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744927 2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:36:54.755185 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744930 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:36:54.755185 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744933 2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:36:54.755185 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744935 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:36:54.755185 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744938 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:36:54.755185 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744940 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:36:54.755185 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744944 2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:36:54.755185 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744947 2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:36:54.755185 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744949 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:36:54.755185 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744952 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:36:54.755185 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744954 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:36:54.755185 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744958 2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:36:54.755649 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744960 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:36:54.755649 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744965 2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:36:54.755649 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744968 2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:36:54.755649 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744970 2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:36:54.755649 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744973 2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:36:54.755649 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744976 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:36:54.755649 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744979 2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:36:54.755649 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744981 2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:36:54.755649 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744984 2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:36:54.755649 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744987 2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:36:54.755649 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744989 2581 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:36:54.755649 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.744992 2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:36:54.755649 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.745018 2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:36:54.755649 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745055 2581 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:36:54.755649 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745060 2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:36:54.755649 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745063 2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:36:54.756025 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745066 2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:36:54.756025 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745068 2581 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:36:54.756025 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745071 2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:36:54.756025 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745073 2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:36:54.756025 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745076 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:36:54.756025 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745079 2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:36:54.756025 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745081 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:36:54.756025 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745085 2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:36:54.756025 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745087 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:36:54.756025 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745090 2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:36:54.756025 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745094 2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:36:54.756025 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745097 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:36:54.756025 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745099 2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:36:54.756025 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745102 2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:36:54.756025 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745105 2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:36:54.756025 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745107 2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:36:54.756025 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745110 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:36:54.756025 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745113 2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:36:54.756025 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745117 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:36:54.756461 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745119 2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:36:54.756461 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745122 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:36:54.756461 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745125 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:36:54.756461 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745128 2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:36:54.756461 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745131 2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:36:54.756461 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745134 2581 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:36:54.756461 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745136 2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:36:54.756461 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745139 2581 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:36:54.756461 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745142 2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:36:54.756461 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745144 2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:36:54.756461 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745147 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:36:54.756461 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745150 2581 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:36:54.756461 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745153 2581 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:36:54.756461 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745155 2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:36:54.756461 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745158 2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:36:54.756461 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745160 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:36:54.756461 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745163 2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:36:54.756461 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745166 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:36:54.756461 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745169 2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:36:54.756461 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745173 2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:36:54.757036 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745177 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:36:54.757036 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745180 2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:36:54.757036 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745182 2581 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:36:54.757036 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745186 2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:36:54.757036 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745189 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:36:54.757036 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745191 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:36:54.757036 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745194 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:36:54.757036 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745197 2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:36:54.757036 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745199 2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:36:54.757036 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745202 2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:36:54.757036 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745204 2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:36:54.757036 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745208 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:36:54.757036 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745211 2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:36:54.757036 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745213 2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:36:54.757036 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745216 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:36:54.757036 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745219 2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:36:54.757036 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745222 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:36:54.757036 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745225 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:36:54.757036 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745227 2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:36:54.757036 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745230 2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:36:54.757521 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745232 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:36:54.757521 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745235 2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:36:54.757521 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745238 2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:36:54.757521 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745242 2581 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:36:54.757521 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745244 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:36:54.757521 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745247 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:36:54.757521 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745249 2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:36:54.757521 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745252 2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:36:54.757521 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745255 2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:36:54.757521 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745258 2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:36:54.757521 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745260 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:36:54.757521 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745263 2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:36:54.757521 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745265 2581 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:36:54.757521 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745268 2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:36:54.757521 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745271 2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:36:54.757521 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745275 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:36:54.757521 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745277 2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:36:54.757521 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745280 2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:36:54.757521 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745284 2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:36:54.758063 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745287 2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:36:54.758063 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745290 2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:36:54.758063 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745293 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:36:54.758063 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745295 2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:36:54.758063 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745299 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:36:54.758063 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745302 2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:36:54.758063 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745305 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:36:54.758063 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745307 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:36:54.758063 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745310 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:36:54.758063 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745313 2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:36:54.758063 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745316 2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:36:54.758063 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745319 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:36:54.758063 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745321 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:36:54.758063 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745324 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:36:54.758063 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745326 2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:36:54.758063 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745329 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:36:54.758063 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745332 2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:36:54.758063 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745335 2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:36:54.758063 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745338 2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:36:54.758553 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745341 2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:36:54.758553 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745343 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:36:54.758553 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745346 2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:36:54.758553 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745348 2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:36:54.758553 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745351 2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:36:54.758553 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.745353 2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:36:54.758553 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.753887 2581 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 18:36:54.758553 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.753905 2581 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 18:36:54.758553 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.753921 2581 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 18:36:54.758553 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.753926 2581 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 18:36:54.758553 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.753990 2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:36:54.758553 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.753995 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:36:54.758553 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.753998 2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:36:54.758553 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754003 2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:36:54.758553 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754007 2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:36:54.758553 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754011 2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:36:54.758553 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754014 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:36:54.758553 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754017 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:36:54.758553 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754019 2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:36:54.759010 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754022 2581 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:36:54.759010 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754026 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:36:54.759010 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754028 2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:36:54.759010 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754031 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:36:54.759010 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754034 2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:36:54.759010 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754037 2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:36:54.759010 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754040 2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:36:54.759010 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754042 2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:36:54.759010 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754045 2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:36:54.759010 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754048 2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:36:54.759010 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754050 2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:36:54.759010 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754053 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:36:54.759010 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754056 2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:36:54.759010 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754058 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:36:54.759010 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754061 2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:36:54.759010 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754064 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:36:54.759010 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754067 2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:36:54.759010 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754069 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:36:54.759010 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754072 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:36:54.759010 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754076 2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:36:54.759476 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754079 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:36:54.759476 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754082 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:36:54.759476 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754085 2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:36:54.759476 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754087 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:36:54.759476 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754090 2581 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:36:54.759476 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754093 2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:36:54.759476 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754095 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:36:54.759476 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754098 2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:36:54.759476 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754100 2581 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:36:54.759476 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754103 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:36:54.759476 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754106 2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:36:54.759476 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754109 2581 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:36:54.759476 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754111 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:36:54.759476 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754114 2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:36:54.759476 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754116 2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:36:54.759476 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754119 2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:36:54.759476 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754122 2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:36:54.759476 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754124 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:36:54.759476 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754127 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:36:54.759476 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754130 2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:36:54.760007 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754133 2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:36:54.760007 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754135 2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:36:54.760007 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754138 2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:36:54.760007 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754140 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:36:54.760007 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754143 2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:36:54.760007 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754146 2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:36:54.760007 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754149 2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:36:54.760007 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754152 2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:36:54.760007 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754156 2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:36:54.760007 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754159 2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:36:54.760007 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754163 2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:36:54.760007 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754167 2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:36:54.760007 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754170 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:36:54.760007 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754173 2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:36:54.760007 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754176 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:36:54.760007 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754179 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:36:54.760007 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754183 2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:36:54.760007 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754185 2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:36:54.760007 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754188 2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:36:54.760471 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754191 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:36:54.760471 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754194 2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:36:54.760471 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754197 2581 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:36:54.760471 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754200 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:36:54.760471 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754202 2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:36:54.760471 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754205 2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:36:54.760471 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754208 2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:36:54.760471 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754211 2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:36:54.760471 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754213 2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:36:54.760471 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754216 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:36:54.760471 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754218 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:36:54.760471 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754222 2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:36:54.760471 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754225 2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:36:54.760471 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754227 2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:36:54.760471 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754230 2581 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:36:54.760471 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754233 2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:36:54.760471 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754235 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:36:54.760471 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754238 2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:36:54.760471 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754241 2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:36:54.760471 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754243 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:36:54.760994 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754246 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:36:54.760994 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754248 2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:36:54.760994 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754251 2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:36:54.760994 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754253 2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:36:54.760994 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754256 2581 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:36:54.760994 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754259 2581 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:36:54.760994 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754261 2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:36:54.760994 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754264 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:36:54.760994 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754267 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:36:54.760994 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754270 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:36:54.760994 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754272 2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:36:54.760994 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754275 2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:36:54.760994 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754278 2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:36:54.760994 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754280 2581 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:36:54.760994 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754283 2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:36:54.760994 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754286 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:36:54.760994 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754288 2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:36:54.760994 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754291 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:36:54.761432 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.754318 2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:36:54.761432 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754463 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:36:54.761432 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754468 2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:36:54.761432 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754471 2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:36:54.761432 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754474 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:36:54.761432 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754477 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:36:54.761432 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754480 2581 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:36:54.761432 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754483 2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:36:54.761432 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754486 2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:36:54.761432 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754489 2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:36:54.761432 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754511 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:36:54.761432 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754515 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:36:54.761432 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754518 2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:36:54.761432 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754520 2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:36:54.761432 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754523 2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:36:54.761432 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754526 2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:36:54.761825 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754528 2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:36:54.761825 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754531 2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:36:54.761825 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754534 2581 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:36:54.761825 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754537 2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:36:54.761825 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754540 2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:36:54.761825 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754543 2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:36:54.761825 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754545 2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:36:54.761825 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754548 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:36:54.761825 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754551 2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:36:54.761825 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754554 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:36:54.761825 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754557 2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:36:54.761825 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754559 2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:36:54.761825 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754562 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:36:54.761825 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754565 2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:36:54.761825 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754568 2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:36:54.761825 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754570 2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:36:54.761825 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754573 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:36:54.761825 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754576 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:36:54.761825 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754578 2581 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:36:54.761825 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754581 2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:36:54.762285 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754583 2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:36:54.762285 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754586 2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:36:54.762285 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754589 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:36:54.762285 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754592 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:36:54.762285 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754595 2581 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:36:54.762285 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754597 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:36:54.762285 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754600 2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:36:54.762285 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754603 2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:36:54.762285 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754605 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:36:54.762285 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754608 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:36:54.762285 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754610 2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:36:54.762285 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754613 2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:36:54.762285 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754615 2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:36:54.762285 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754618 2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:36:54.762285 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754620 2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:36:54.762285 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754624 2581 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:36:54.762285 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754626 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:36:54.762285 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754629 2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:36:54.762285 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754632 2581 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:36:54.762285 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754634 2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:36:54.762782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754637 2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:36:54.762782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754640 2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:36:54.762782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754642 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:36:54.762782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754645 2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:36:54.762782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754648 2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:36:54.762782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754653 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:36:54.762782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754657 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:36:54.762782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754662 2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:36:54.762782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754666 2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:36:54.762782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754670 2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:36:54.762782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754673 2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:36:54.762782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754676 2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:36:54.762782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754680 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:36:54.762782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754682 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:36:54.762782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754685 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:36:54.762782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754688 2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:36:54.762782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754692 2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:36:54.762782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754696 2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:36:54.762782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754699 2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:36:54.763253 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754702 2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:36:54.763253 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754705 2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:36:54.763253 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754708 2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:36:54.763253 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754711 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:36:54.763253 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754713 2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:36:54.763253 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754715 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:36:54.763253 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754718 2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:36:54.763253 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754721 2581 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:36:54.763253 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754724 2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:36:54.763253 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754726 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:36:54.763253 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754729 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:36:54.763253 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754731 2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:36:54.763253 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754734 2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:36:54.763253 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754736 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:36:54.763253 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754739 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:36:54.763253 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754743 2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:36:54.763253 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754747 2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:36:54.763253 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754750 2581 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:36:54.763253 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754753 2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:36:54.763782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754756 2581 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:36:54.763782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754759 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:36:54.763782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754762 2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:36:54.763782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754764 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:36:54.763782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754767 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:36:54.763782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754769 2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:36:54.763782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754772 2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:36:54.763782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754774 2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:36:54.763782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754776 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:36:54.763782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754779 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:36:54.763782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754781 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:36:54.763782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754785 2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:36:54.763782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754787 2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:36:54.763782 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.754811 2581 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:36:54.763782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754845 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:36:54.763782 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754850 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:36:54.764147 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754853 2581 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:36:54.764147 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754856 2581 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:36:54.764147 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754859 2581 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:36:54.764147 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754861 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:36:54.764147 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754864 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:36:54.764147 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754867 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:36:54.764147 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754869 2581 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:36:54.764147 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754872 2581 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:36:54.764147 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754875 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:36:54.764147 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754877 2581 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:36:54.764147 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754880 2581 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:36:54.764147 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754883 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:36:54.764147 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754885 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:36:54.764147 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754888 2581 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:36:54.764147 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754891 2581 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:36:54.764147 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754893 2581 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:36:54.764147 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754896 2581 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:36:54.764147 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754898 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:36:54.764147 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754901 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:36:54.764147 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754903 2581 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:36:54.764664 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754906 2581 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:36:54.764664 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754908 2581 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:36:54.764664 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754911 2581 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:36:54.764664 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754913 2581 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:36:54.764664 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754916 2581 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:36:54.764664 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754918 2581 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:36:54.764664 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754921 2581 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:36:54.764664 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754924 2581 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:36:54.764664 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754928 2581 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:36:54.764664 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754933 2581 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:36:54.764664 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754935 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:36:54.764664 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754938 2581 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:36:54.764664 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754941 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:36:54.764664 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754943 2581 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:36:54.764664 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754946 2581 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:36:54.764664 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754948 2581 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:36:54.764664 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754951 2581 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:36:54.764664 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754953 2581 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:36:54.764664 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754956 2581 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:36:54.764664 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754959 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:36:54.765146 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754962 2581 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:36:54.765146 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754964 2581 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:36:54.765146 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754967 2581 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:36:54.765146 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754970 2581 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:36:54.765146 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754972 2581 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:36:54.765146 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754975 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:36:54.765146 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754977 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:36:54.765146 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754980 2581 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:36:54.765146 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754983 2581 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:36:54.765146 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754986 2581 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:36:54.765146 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754988 2581 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:36:54.765146 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754991 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:36:54.765146 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754994 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:36:54.765146 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754996 2581 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:36:54.765146 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.754999 2581 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:36:54.765146 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755001 2581 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:36:54.765146 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755004 2581 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:36:54.765146 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755007 2581 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:36:54.765146 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755009 2581 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:36:54.765146 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755012 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:36:54.765624 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755015 2581 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:36:54.765624 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755025 2581 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:36:54.765624 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755029 2581 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:36:54.765624 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755032 2581 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:36:54.765624 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755034 2581 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:36:54.765624 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755037 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:36:54.765624 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755039 2581 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:36:54.765624 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755042 2581 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:36:54.765624 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755045 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:36:54.765624 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755047 2581 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:36:54.765624 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755050 2581 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:36:54.765624 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755052 2581 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:36:54.765624 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755055 2581 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:36:54.765624 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755058 2581 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:36:54.765624 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755061 2581 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:36:54.765624 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755063 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:36:54.765624 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755066 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:36:54.765624 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755068 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:36:54.765624 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755071 2581 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:36:54.765624 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755073 2581 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:36:54.766086 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755077 2581 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:36:54.766086 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755081 2581 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:36:54.766086 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755084 2581 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:36:54.766086 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755086 2581 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:36:54.766086 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755088 2581 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:36:54.766086 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755091 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:36:54.766086 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755093 2581 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:36:54.766086 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755096 2581 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:36:54.766086 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755098 2581 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:36:54.766086 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755101 2581 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:36:54.766086 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755104 2581 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:36:54.766086 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755106 2581 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:36:54.766086 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755109 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:36:54.766086 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755113 2581 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:36:54.766086 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755116 2581 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:36:54.766086 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755119 2581 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:36:54.766086 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755121 2581 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:36:54.766086 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755125 2581 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:36:54.766515 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755129 2581 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:36:54.766515 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755132 2581 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:36:54.766515 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755134 2581 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:36:54.766515 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755137 2581 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:36:54.766515 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755140 2581 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:36:54.766515 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:54.755143 2581 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:36:54.766515 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.755225 2581 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 18:36:54.766515 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.757645 2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Oct 30 18:36:54.766515 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.758635 2581 server.go:1014] "Starting client certificate rotation" Oct 30 18:36:54.766515 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.758724 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 18:36:54.766515 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.758757 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 18:36:54.766515 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.760901 2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:36:54.766515 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.762037 2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:36:54.773695 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.773680 2581 log.go:25] "Validated CRI v1 runtime API" Oct 30 18:36:54.777804 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.777792 2581 log.go:25] "Validated CRI v1 image API" Oct 30 18:36:54.778617 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.778593 2581 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 18:36:54.785013 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.784986 2581 fs.go:135] Filesystem UUIDs: map[20945065-5c2f-495f-a16a-d494deccdd9a:/dev/nvme0n1p4 428c119f-d882-4cfe-bf7f-8579b734ec25:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Oct 30 18:36:54.785084 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.785011 2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Oct 30 18:36:54.791683 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.791568 2581 manager.go:217] Machine: {Timestamp:2026-10-30 18:36:54.7898073 +0000 UTC m=+0.375862737 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3107591 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ba733113a161fbb88a5b528f0ba92 SystemUUID:ec2ba733-113a-161f-bb88-a5b528f0ba92 BootID:a546173e-5f71-4725-bb31-5f4119027579 Filesystems:[{Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:ce:7e:29:71 Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:ce:7e:29:71 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:66:39:97:1c:72:39 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 18:36:54.791683 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.791678 2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 18:36:54.791831 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.791756 2581 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 18:36:54.793051 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.793019 2581 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 18:36:54.793239 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.793053 2581 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-74.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 18:36:54.793323 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.793243 2581 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 18:36:54.793323 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.793256 2581 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 18:36:54.793323 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.793308 2581 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:36:54.793427 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.793334 2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:36:54.794566 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.794553 2581 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 18:36:54.799183 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.799169 2581 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 18:36:54.799306 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.799295 2581 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 18:36:54.800164 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.800153 2581 kubelet.go:493] "Attempting to sync node with API server" Oct 30 18:36:54.800225 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.800170 2581 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 18:36:54.800225 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.800184 2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 18:36:54.800225 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.800198 2581 kubelet.go:405] "Adding apiserver pod source" Oct 30 18:36:54.800225 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.800209 2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 18:36:54.800547 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.800534 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:36:54.803034 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.803010 2581 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 18:36:54.809914 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.809902 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:36:54.810834 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.810822 2581 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 18:36:54.810872 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.810852 2581 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 18:36:54.811006 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.810995 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 18:36:54.811040 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.811011 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 18:36:54.811040 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.811017 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 18:36:54.811040 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.811022 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 18:36:54.811040 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.811027 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 18:36:54.811040 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.811033 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 18:36:54.811762 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.811753 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 18:36:54.811798 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.811763 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 18:36:54.811798 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.811770 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 18:36:54.811798 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.811782 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 18:36:54.811798 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.811795 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 18:36:54.811966 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.811805 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 18:36:54.812596 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.812586 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 18:36:54.812596 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.812596 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 18:36:54.816343 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.816321 2581 server.go:1263] "Started kubelet" Oct 30 18:36:54.816416 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.816392 2581 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 18:36:54.816548 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.816480 2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 18:36:54.816582 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.816570 2581 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 18:36:54.817857 ip-10-0-132-74 systemd[1]: Started Kubernetes Kubelet. Oct 30 18:36:54.818994 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.817963 2581 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 18:36:54.821233 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.821216 2581 server.go:317] "Adding debug handlers to kubelet server" Oct 30 18:36:54.824589 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:54.824567 2581 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Oct 30 18:36:54.826413 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.826392 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 18:36:54.827140 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.827123 2581 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 18:36:54.827314 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:54.827293 2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-74.us-west-2.compute.internal\" not found" Oct 30 18:36:54.827978 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.827961 2581 factory.go:55] Registering systemd factory Oct 30 18:36:54.827978 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.827979 2581 factory.go:223] Registration of the systemd container factory successfully Oct 30 18:36:54.828133 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.828116 2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 18:36:54.828204 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.828124 2581 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 18:36:54.828204 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.828146 2581 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 18:36:54.828204 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.828183 2581 factory.go:153] Registering CRI-O factory Oct 30 18:36:54.828204 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.828197 2581 factory.go:223] Registration of the crio container factory successfully Oct 30 18:36:54.828354 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.828253 2581 reconstruct.go:97] "Volume reconstruction finished" Oct 30 18:36:54.828354 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.828267 2581 reconciler.go:29] "Reconciler: start to sync state" Oct 30 18:36:54.828354 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.828255 2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 18:36:54.828354 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.828288 2581 factory.go:103] Registering Raw factory Oct 30 18:36:54.828354 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.828298 2581 manager.go:1199] Started watching for new ooms in manager Oct 30 18:36:54.829278 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.829265 2581 manager.go:319] Starting recovery of all containers Oct 30 18:36:54.840385 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.840365 2581 manager.go:324] Recovery completed Oct 30 18:36:54.844889 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.844876 2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:36:54.846647 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.846620 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:36:54.846704 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.846663 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:36:54.846704 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.846679 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasSufficientPID" Oct 30 18:36:54.847180 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.847162 2581 cpu_manager.go:226] "Starting" policy="none" Oct 30 18:36:54.847180 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.847178 2581 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 18:36:54.847268 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.847194 2581 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 18:36:54.847268 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.847168 2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:36:54.847855 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.847840 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:36:54.847914 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.847868 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:36:54.847914 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.847881 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasSufficientPID" Oct 30 18:36:54.849352 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.849338 2581 policy_none.go:50] "Start" Oct 30 18:36:54.849352 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.849353 2581 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 18:36:54.849427 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.849362 2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 18:36:54.851327 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.851317 2581 policy_none.go:44] "Start" Oct 30 18:36:54.876973 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.876941 2581 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 18:36:54.879582 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.879565 2581 manager.go:341] "Starting Device Plugin manager" Oct 30 18:36:54.879668 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:54.879596 2581 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 18:36:54.879668 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.879607 2581 server.go:87] "Starting device plugin registration server" Oct 30 18:36:54.879844 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.879816 2581 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 18:36:54.879844 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.879828 2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 18:36:54.880099 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.879954 2581 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 18:36:54.880099 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.880022 2581 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 18:36:54.880099 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.880029 2581 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 18:36:54.880873 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:54.880843 2581 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 18:36:54.880952 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:54.880897 2581 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-74.us-west-2.compute.internal\" not found" Oct 30 18:36:54.914009 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.904721 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 18:36:54.914009 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.905855 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 18:36:54.914009 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.905874 2581 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 18:36:54.914009 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.905893 2581 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 18:36:54.914009 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:54.905923 2581 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Oct 30 18:36:54.952555 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.952515 2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-74.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Oct 30 18:36:54.952555 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:54.952521 2581 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-74.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Oct 30 18:36:54.953676 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:54.952523 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-74.us-west-2.compute.internal.18e36288737ded7e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-74.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-74.us-west-2.compute.internal,},FirstTimestamp:2026-10-30 18:36:54.816296318 +0000 UTC m=+0.402351754,LastTimestamp:2026-10-30 18:36:54.816296318 +0000 UTC m=+0.402351754,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-74.us-west-2.compute.internal,}" Oct 30 18:36:54.980102 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.980088 2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:36:54.980950 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.980933 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:36:54.981022 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.980964 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:36:54.981022 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.980984 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasSufficientPID" Oct 30 18:36:54.981022 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:54.981007 2581 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:36:55.006937 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.006773 2581 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal"] Oct 30 18:36:55.007001 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.006944 2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:36:55.007735 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.007715 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:36:55.007805 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.007739 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:36:55.007805 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.007749 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasSufficientPID" Oct 30 18:36:55.009962 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.009951 2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:36:55.010103 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.010090 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:36:55.010139 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.010119 2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:36:55.010673 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.010658 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:36:55.010673 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.010670 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:36:55.010776 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.010686 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:36:55.010776 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.010691 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:36:55.010776 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.010698 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasSufficientPID" Oct 30 18:36:55.010776 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.010701 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasSufficientPID" Oct 30 18:36:55.012030 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:55.011960 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-74.us-west-2.compute.internal.18e36288754d0a7d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-74.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-74.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-74.us-west-2.compute.internal,},FirstTimestamp:2026-10-30 18:36:54.846646909 +0000 UTC m=+0.432702350,LastTimestamp:2026-10-30 18:36:54.846646909 +0000 UTC m=+0.432702350,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-74.us-west-2.compute.internal,}" Oct 30 18:36:55.012149 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:55.012116 2581 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" Oct 30 18:36:55.012469 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:55.012448 2581 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-74.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" Oct 30 18:36:55.012813 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.012795 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:36:55.012860 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.012822 2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:36:55.013447 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.013431 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:36:55.013523 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.013456 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:36:55.013523 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.013465 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasSufficientPID" Oct 30 18:36:55.027036 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.027018 2581 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 18:36:55.040971 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:55.040957 2581 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-74.us-west-2.compute.internal\" not found" node="ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:36:55.043379 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:55.043357 2581 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-74.us-west-2.compute.internal\" not found" node="ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:36:55.058675 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:55.058659 2581 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" Oct 30 18:36:55.084413 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.084394 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z2mlj" Oct 30 18:36:55.095486 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.095467 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-z2mlj" Oct 30 18:36:55.106189 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:55.106146 2581 reflector.go:204] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" Oct 30 18:36:55.106270 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:55.106205 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-74.us-west-2.compute.internal.18e36288754d68b7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-74.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-74.us-west-2.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-74.us-west-2.compute.internal,},FirstTimestamp:2026-10-30 18:36:54.846671031 +0000 UTC m=+0.432726471,LastTimestamp:2026-10-30 18:36:54.846671031 +0000 UTC m=+0.432726471,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-74.us-west-2.compute.internal,}" Oct 30 18:36:55.129224 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.129203 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a99a11e235547936507bfb99804f96f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal\" (UID: \"6a99a11e235547936507bfb99804f96f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:36:55.129306 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.129229 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a99a11e235547936507bfb99804f96f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal\" (UID: \"6a99a11e235547936507bfb99804f96f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:36:55.129306 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.129249 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/958b813bafdd4852b0fbfdc2a07cf4ae-config\") pod \"kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal\" (UID: \"958b813bafdd4852b0fbfdc2a07cf4ae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:36:55.172031 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.172012 2581 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:36:55.172101 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:55.172032 2581 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-74.us-west-2.compute.internal\": node \"ip-10-0-132-74.us-west-2.compute.internal\" not found" Oct 30 18:36:55.230088 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.230060 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a99a11e235547936507bfb99804f96f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal\" (UID: \"6a99a11e235547936507bfb99804f96f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:36:55.230200 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.230090 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a99a11e235547936507bfb99804f96f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal\" (UID: \"6a99a11e235547936507bfb99804f96f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:36:55.230200 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.230105 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/958b813bafdd4852b0fbfdc2a07cf4ae-config\") pod \"kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal\" (UID: \"958b813bafdd4852b0fbfdc2a07cf4ae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:36:55.230200 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.230149 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/958b813bafdd4852b0fbfdc2a07cf4ae-config\") pod \"kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal\" (UID: \"958b813bafdd4852b0fbfdc2a07cf4ae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:36:55.230200 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.230165 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a99a11e235547936507bfb99804f96f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal\" (UID: \"6a99a11e235547936507bfb99804f96f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:36:55.230200 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.230171 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a99a11e235547936507bfb99804f96f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal\" (UID: \"6a99a11e235547936507bfb99804f96f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:36:55.282385 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:55.282334 2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-74.us-west-2.compute.internal\" not found" Oct 30 18:36:55.344583 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.344561 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:36:55.350014 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:55.349537 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:36:55.383039 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:55.383022 2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-74.us-west-2.compute.internal\" not found" Oct 30 18:36:55.484018 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:55.483987 2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-74.us-west-2.compute.internal\" not found" Oct 30 18:36:55.584963 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:55.584887 2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-74.us-west-2.compute.internal\" not found" Oct 30 18:36:56.960691 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:56.960664 2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-74.us-west-2.compute.internal\" not found" Oct 30 18:36:57.033492 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:57.033467 2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Oct 30 18:36:57.033847 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:57.033589 2581 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a1170fdc21ac14235b67aafb8b26d5e9-34204dc823c30344.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-132-74.us-west-2.compute.internal.18e36288754d0a7d\": read tcp 10.0.132.74:32958->3.143.200.93:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-132-74.us-west-2.compute.internal.18e36288754d0a7d default 4740 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-74.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-74.us-west-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-74.us-west-2.compute.internal,},FirstTimestamp:2026-10-30 18:36:54 +0000 UTC,LastTimestamp:2026-10-30 18:36:55.007729389 +0000 UTC m=+0.593784823,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-74.us-west-2.compute.internal,}" Oct 30 18:36:57.061793 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:57.061757 2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-74.us-west-2.compute.internal\" not found" Oct 30 18:36:57.162700 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:57.162635 2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-74.us-west-2.compute.internal\" not found" Oct 30 18:36:57.240586 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:57.240557 2581 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:36:57.241586 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:57.241571 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:36:57.241659 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:57.241602 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:36:57.241659 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:57.241613 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasSufficientPID" Oct 30 18:36:57.263222 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:57.263201 2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-74.us-west-2.compute.internal\" not found" Oct 30 18:36:57.363683 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:57.363662 2581 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-74.us-west-2.compute.internal\" not found" Oct 30 18:36:57.365340 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:57.365324 2581 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:36:57.376741 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:57.376714 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:31:56 +0000 UTC" deadline="2028-06-16 14:29:03.697850121 +0000 UTC" Oct 30 18:36:57.376741 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:57.376739 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14275h52m6.321115034s" Oct 30 18:36:57.403899 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:57.403879 2581 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:36:57.413103 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:57.413050 2581 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:36:57.467653 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:57.467636 2581 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:36:57.468488 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:57.468475 2581 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:36:57.523621 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:57.523601 2581 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:36:57.559082 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:57.559055 2581 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:36:57.559187 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:57.559115 2581 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:36:57.708879 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:57.708837 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a99a11e235547936507bfb99804f96f.slice/crio-0e9cdc0cc1bebc95d5b3ec8adbeec0beee23bf505befc959d302298857d4a3fc WatchSource:0}: Error finding container 0e9cdc0cc1bebc95d5b3ec8adbeec0beee23bf505befc959d302298857d4a3fc: Status 404 returned error can't find the container with id 0e9cdc0cc1bebc95d5b3ec8adbeec0beee23bf505befc959d302298857d4a3fc Oct 30 18:36:57.709335 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:57.709310 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod958b813bafdd4852b0fbfdc2a07cf4ae.slice/crio-558cbff2e8507037d2164089aed5694c6267f1ddbf88050df2e780fdbc0bec89 WatchSource:0}: Error finding container 558cbff2e8507037d2164089aed5694c6267f1ddbf88050df2e780fdbc0bec89: Status 404 returned error can't find the container with id 558cbff2e8507037d2164089aed5694c6267f1ddbf88050df2e780fdbc0bec89 Oct 30 18:36:57.715064 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:57.715044 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:36:58.076956 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.076887 2581 apiserver.go:52] "Watching apiserver" Oct 30 18:36:58.103065 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.103038 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Oct 30 18:36:58.175575 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.175550 2581 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 18:36:58.175854 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.175831 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv","openshift-cluster-node-tuning-operator/tuned-fbfdv","openshift-multus/network-metrics-daemon-phngs","kube-system/global-pull-secret-syncer-pxndl","kube-system/konnectivity-agent-w2wtr","openshift-image-registry/node-ca-mxjc4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal","openshift-multus/multus-additional-cni-plugins-7dkzl","openshift-network-diagnostics/network-check-target-62vfn","openshift-ovn-kubernetes/ovnkube-node-4hsc7","kube-system/kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal","openshift-dns/node-resolver-k7r7n","openshift-multus/multus-928bp","openshift-network-operator/iptables-alerter-p88j5"] Oct 30 18:36:58.179665 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.179646 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.181809 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.181787 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-928bp" Oct 30 18:36:58.184028 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.183996 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:36:58.184110 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:58.184083 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-phngs" podUID="25bce85b-a92c-4c39-a298-7b1881380069" Oct 30 18:36:58.186857 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.186835 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:36:58.186974 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:58.186935 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pxndl" podUID="f1cae505-4cef-46e5-85cc-cf8b410e654a" Oct 30 18:36:58.189115 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.189100 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w2wtr" Oct 30 18:36:58.192286 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.192266 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mxjc4" Oct 30 18:36:58.194444 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.194429 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:36:58.194531 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:58.194489 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62vfn" podUID="97294b18-463b-429d-8073-dc532ab0a046" Oct 30 18:36:58.196482 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.196467 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p88j5" Oct 30 18:36:58.198688 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.198669 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.200907 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.200891 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.203223 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.203208 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.205520 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.205503 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k7r7n" Oct 30 18:36:58.208261 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.208223 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" event={"ID":"6a99a11e235547936507bfb99804f96f","Type":"ContainerStarted","Data":"0e9cdc0cc1bebc95d5b3ec8adbeec0beee23bf505befc959d302298857d4a3fc"} Oct 30 18:36:58.208333 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.208272 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal" event={"ID":"958b813bafdd4852b0fbfdc2a07cf4ae","Type":"ContainerStarted","Data":"558cbff2e8507037d2164089aed5694c6267f1ddbf88050df2e780fdbc0bec89"} Oct 30 18:36:58.214357 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.214339 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-run-openvswitch\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.214430 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.214362 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-log-socket\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.214430 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.214380 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-run-ovn-kubernetes\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.214430 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.214414 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-cni-bin\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.214546 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.214510 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/70781b52-75b2-450a-a058-b0d609659cda-ovnkube-config\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.214590 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.214560 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/70781b52-75b2-450a-a058-b0d609659cda-env-overrides\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.214632 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.214613 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/70781b52-75b2-450a-a058-b0d609659cda-ovn-node-metrics-cert\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.214681 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.214631 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a29b358-3cb7-49e6-85a9-6581a2f52bd1-host\") pod \"node-ca-mxjc4\" (UID: \"6a29b358-3cb7-49e6-85a9-6581a2f52bd1\") " pod="openshift-image-registry/node-ca-mxjc4" Oct 30 18:36:58.214725 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.214685 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-var-lib-cni-multus\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.214802 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.214739 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-hostroot\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.214802 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.214757 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-host\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.214802 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.214776 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-device-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.214894 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.214813 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-system-cni-dir\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.214894 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.214831 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5tc8\" (UniqueName: \"kubernetes.io/projected/d8e44bd9-13d6-4e54-9d9b-bfad33e1393c-kube-api-access-l5tc8\") pod \"node-resolver-k7r7n\" (UID: \"d8e44bd9-13d6-4e54-9d9b-bfad33e1393c\") " pod="openshift-dns/node-resolver-k7r7n" Oct 30 18:36:58.214894 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.214852 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tww25\" (UniqueName: \"kubernetes.io/projected/70781b52-75b2-450a-a058-b0d609659cda-kube-api-access-tww25\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.214894 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.214876 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr22p\" (UniqueName: \"kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p\") pod \"network-check-target-62vfn\" (UID: \"97294b18-463b-429d-8073-dc532ab0a046\") " pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:36:58.214894 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.214891 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-system-cni-dir\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.215148 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.214911 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-systemd\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.215148 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.214952 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-tuned\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.215148 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.214991 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-cni-netd\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.215148 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215045 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1bec85c3-72c2-46dc-9f68-8e80f1478422-agent-certs\") pod \"konnectivity-agent-w2wtr\" (UID: \"1bec85c3-72c2-46dc-9f68-8e80f1478422\") " pod="kube-system/konnectivity-agent-w2wtr" Oct 30 18:36:58.215148 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215085 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhpbc\" (UniqueName: \"kubernetes.io/projected/10a7e4f4-2efc-4803-b3c2-da8749031462-kube-api-access-nhpbc\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.215148 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215110 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-var-lib-kubelet\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.215148 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215133 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-etc-selinux\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.215412 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215172 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-run-netns\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.215412 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215212 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs\") pod \"network-metrics-daemon-phngs\" (UID: \"25bce85b-a92c-4c39-a298-7b1881380069\") " pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:36:58.215412 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215231 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-socket-dir-parent\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.215412 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215253 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-sysctl-conf\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.215412 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215270 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7n2h\" (UniqueName: \"kubernetes.io/projected/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-kube-api-access-n7n2h\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.215412 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215290 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-node-log\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.215412 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215311 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbb9x\" (UniqueName: \"kubernetes.io/projected/6a29b358-3cb7-49e6-85a9-6581a2f52bd1-kube-api-access-gbb9x\") pod \"node-ca-mxjc4\" (UID: \"6a29b358-3cb7-49e6-85a9-6581a2f52bd1\") " pod="openshift-image-registry/node-ca-mxjc4" Oct 30 18:36:58.215412 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215356 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-kubernetes\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.215412 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215381 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-lib-modules\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.215412 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215409 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-kubelet\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.215821 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215427 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/70781b52-75b2-450a-a058-b0d609659cda-ovnkube-script-lib\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.215821 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215450 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxznv\" (UniqueName: \"kubernetes.io/projected/25bce85b-a92c-4c39-a298-7b1881380069-kube-api-access-fxznv\") pod \"network-metrics-daemon-phngs\" (UID: \"25bce85b-a92c-4c39-a298-7b1881380069\") " pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:36:58.215821 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215465 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1bec85c3-72c2-46dc-9f68-8e80f1478422-konnectivity-ca\") pod \"konnectivity-agent-w2wtr\" (UID: \"1bec85c3-72c2-46dc-9f68-8e80f1478422\") " pod="kube-system/konnectivity-agent-w2wtr" Oct 30 18:36:58.215821 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215508 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-sysconfig\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.215821 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215539 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-sys\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.215821 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215565 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-registration-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.215821 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215587 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.215821 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215604 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6a29b358-3cb7-49e6-85a9-6581a2f52bd1-serviceca\") pod \"node-ca-mxjc4\" (UID: \"6a29b358-3cb7-49e6-85a9-6581a2f52bd1\") " pod="openshift-image-registry/node-ca-mxjc4" Oct 30 18:36:58.215821 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215652 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-run-multus-certs\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.215821 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215693 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8pcp\" (UniqueName: \"kubernetes.io/projected/8a3c8868-93e2-4f20-910e-d2cd73bc1d5f-kube-api-access-v8pcp\") pod \"iptables-alerter-p88j5\" (UID: \"8a3c8868-93e2-4f20-910e-d2cd73bc1d5f\") " pod="openshift-network-operator/iptables-alerter-p88j5" Oct 30 18:36:58.215821 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215737 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-sysctl-d\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.215821 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215762 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-modprobe-d\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.215821 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215786 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.216223 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215828 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8e44bd9-13d6-4e54-9d9b-bfad33e1393c-tmp-dir\") pod \"node-resolver-k7r7n\" (UID: \"d8e44bd9-13d6-4e54-9d9b-bfad33e1393c\") " pod="openshift-dns/node-resolver-k7r7n" Oct 30 18:36:58.216223 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215849 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-systemd-units\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.216223 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215868 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:36:58.216223 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215905 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-cnibin\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.216223 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.215942 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8443eba-410b-4aea-94b7-f5fdabbb06e3-tmp\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.216223 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216051 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q64j\" (UniqueName: \"kubernetes.io/projected/d8443eba-410b-4aea-94b7-f5fdabbb06e3-kube-api-access-9q64j\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.216223 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216079 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.216223 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216093 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-cnibin\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.216223 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216115 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-run-systemd\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.216223 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216155 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-etc-openvswitch\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.216223 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216175 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-run-ovn\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.216223 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216195 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f1cae505-4cef-46e5-85cc-cf8b410e654a-dbus\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:36:58.216578 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216229 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8a3c8868-93e2-4f20-910e-d2cd73bc1d5f-iptables-alerter-script\") pod \"iptables-alerter-p88j5\" (UID: \"8a3c8868-93e2-4f20-910e-d2cd73bc1d5f\") " pod="openshift-network-operator/iptables-alerter-p88j5" Oct 30 18:36:58.216578 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216262 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-run\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.216578 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216295 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-sys-fs\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.216578 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216311 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-var-lib-openvswitch\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.216578 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216333 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-var-lib-cni-bin\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.216578 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216355 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.216578 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216394 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f1cae505-4cef-46e5-85cc-cf8b410e654a-kubelet-config\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:36:58.216578 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216434 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-cni-dir\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.216578 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216470 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/10a7e4f4-2efc-4803-b3c2-da8749031462-cni-binary-copy\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.216578 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216494 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-run-k8s-cni-cncf-io\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.216578 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216522 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-etc-kubernetes\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.216578 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216543 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.216971 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216585 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-daemon-config\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.216971 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216627 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a3c8868-93e2-4f20-910e-d2cd73bc1d5f-host-slash\") pod \"iptables-alerter-p88j5\" (UID: \"8a3c8868-93e2-4f20-910e-d2cd73bc1d5f\") " pod="openshift-network-operator/iptables-alerter-p88j5" Oct 30 18:36:58.216971 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216645 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-socket-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.216971 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216685 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftcrx\" (UniqueName: \"kubernetes.io/projected/b7ad340e-8e80-4f55-a29b-e8d501946cb5-kube-api-access-ftcrx\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.216971 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216716 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-run-netns\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.216971 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216741 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-conf-dir\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.216971 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216765 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-slash\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.216971 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216790 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-var-lib-kubelet\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.216971 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216818 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-cni-binary-copy\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.216971 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.216852 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d8e44bd9-13d6-4e54-9d9b-bfad33e1393c-hosts-file\") pod \"node-resolver-k7r7n\" (UID: \"d8e44bd9-13d6-4e54-9d9b-bfad33e1393c\") " pod="openshift-dns/node-resolver-k7r7n" Oct 30 18:36:58.232327 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.232306 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 18:36:58.232447 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.232370 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 18:36:58.232500 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.232455 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 18:36:58.232635 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.232620 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 18:36:58.232773 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.232759 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 18:36:58.233069 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.233057 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 18:36:58.233123 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.233114 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 18:36:58.233217 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.233200 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 18:36:58.233376 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.233364 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 18:36:58.233620 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.233589 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 18:36:58.234036 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.234019 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-lp85d\"" Oct 30 18:36:58.234139 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.234121 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gh5g8\"" Oct 30 18:36:58.239120 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.239103 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 18:36:58.239760 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.239745 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 18:36:58.271873 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.271853 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 18:36:58.272825 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.272810 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-j99tc\"" Oct 30 18:36:58.278611 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.278596 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 18:36:58.278831 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.278818 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9cgrs\"" Oct 30 18:36:58.278869 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.278860 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 18:36:58.278901 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.278884 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 18:36:58.279396 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.279379 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-czdz5\"" Oct 30 18:36:58.279445 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.279432 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lfj5d\"" Oct 30 18:36:58.279481 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.279476 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4rg2h\"" Oct 30 18:36:58.279520 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.279498 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 18:36:58.279553 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.279517 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jqqcl\"" Oct 30 18:36:58.279553 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.279520 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hlrng\"" Oct 30 18:36:58.279929 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.279913 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 18:36:58.279929 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.279917 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 18:36:58.280060 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.279928 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 18:36:58.280060 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.279930 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 18:36:58.280168 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.280154 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 18:36:58.280222 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.280188 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 18:36:58.280222 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.280204 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:36:58.280300 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.280238 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:36:58.280300 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.280252 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 18:36:58.280300 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.280257 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 18:36:58.280300 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.280258 2581 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 18:36:58.280434 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.280300 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qqc65" Oct 30 18:36:58.304299 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.304283 2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 18:36:58.317677 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.317656 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.317741 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.317680 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6a29b358-3cb7-49e6-85a9-6581a2f52bd1-serviceca\") pod \"node-ca-mxjc4\" (UID: \"6a29b358-3cb7-49e6-85a9-6581a2f52bd1\") " pod="openshift-image-registry/node-ca-mxjc4" Oct 30 18:36:58.317741 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.317699 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-run-multus-certs\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.317741 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.317720 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8pcp\" (UniqueName: \"kubernetes.io/projected/8a3c8868-93e2-4f20-910e-d2cd73bc1d5f-kube-api-access-v8pcp\") pod \"iptables-alerter-p88j5\" (UID: \"8a3c8868-93e2-4f20-910e-d2cd73bc1d5f\") " pod="openshift-network-operator/iptables-alerter-p88j5" Oct 30 18:36:58.317741 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.317735 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-sysctl-d\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.317918 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.317749 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-modprobe-d\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.317918 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.317765 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.317918 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.317768 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.317918 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.317794 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-run-multus-certs\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.317918 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.317819 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8e44bd9-13d6-4e54-9d9b-bfad33e1393c-tmp-dir\") pod \"node-resolver-k7r7n\" (UID: \"d8e44bd9-13d6-4e54-9d9b-bfad33e1393c\") " pod="openshift-dns/node-resolver-k7r7n" Oct 30 18:36:58.317918 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.317847 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-systemd-units\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.317918 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.317872 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:36:58.317918 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.317899 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-modprobe-d\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.317918 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.317918 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-sysctl-d\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.318346 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.317898 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-cnibin\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.318346 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.317946 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-systemd-units\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.318346 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.317963 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8443eba-410b-4aea-94b7-f5fdabbb06e3-tmp\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.318346 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.317986 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q64j\" (UniqueName: \"kubernetes.io/projected/d8443eba-410b-4aea-94b7-f5fdabbb06e3-kube-api-access-9q64j\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.318346 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.317996 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-cnibin\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.318346 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318026 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.318346 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318051 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-cnibin\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.318346 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318074 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-run-systemd\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.318346 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:58.318028 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:36:58.318346 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318099 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-etc-openvswitch\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.318346 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318120 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-run-ovn\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.318346 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318140 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f1cae505-4cef-46e5-85cc-cf8b410e654a-dbus\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:36:58.318346 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318165 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8a3c8868-93e2-4f20-910e-d2cd73bc1d5f-iptables-alerter-script\") pod \"iptables-alerter-p88j5\" (UID: \"8a3c8868-93e2-4f20-910e-d2cd73bc1d5f\") " pod="openshift-network-operator/iptables-alerter-p88j5" Oct 30 18:36:58.318346 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:58.318204 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret podName:f1cae505-4cef-46e5-85cc-cf8b410e654a nodeName:}" failed. No retries permitted until 2026-10-30 18:36:58.818180846 +0000 UTC m=+3.128726750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret") pod "global-pull-secret-syncer-pxndl" (UID: "f1cae505-4cef-46e5-85cc-cf8b410e654a") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:36:58.318346 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318226 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-run\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.318346 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318238 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6a29b358-3cb7-49e6-85a9-6581a2f52bd1-serviceca\") pod \"node-ca-mxjc4\" (UID: \"6a29b358-3cb7-49e6-85a9-6581a2f52bd1\") " pod="openshift-image-registry/node-ca-mxjc4" Oct 30 18:36:58.318346 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318259 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-sys-fs\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.319150 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318287 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-var-lib-openvswitch\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.319150 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318294 2581 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 18:36:58.319150 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318308 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-cnibin\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.319150 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318312 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-var-lib-cni-bin\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.319150 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318338 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-run-ovn\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.319150 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318349 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.319150 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318359 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.319150 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318296 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.319150 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318354 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-var-lib-cni-bin\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.319150 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318352 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8e44bd9-13d6-4e54-9d9b-bfad33e1393c-tmp-dir\") pod \"node-resolver-k7r7n\" (UID: \"d8e44bd9-13d6-4e54-9d9b-bfad33e1393c\") " pod="openshift-dns/node-resolver-k7r7n" Oct 30 18:36:58.319150 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318405 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-var-lib-openvswitch\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.319150 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318407 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-sys-fs\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.319150 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318323 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-run-systemd\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.319150 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318477 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f1cae505-4cef-46e5-85cc-cf8b410e654a-kubelet-config\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:36:58.319150 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318503 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-cni-dir\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.319150 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318520 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-run\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.319150 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318515 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f1cae505-4cef-46e5-85cc-cf8b410e654a-dbus\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:36:58.319150 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318547 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/10a7e4f4-2efc-4803-b3c2-da8749031462-cni-binary-copy\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.320059 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318565 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f1cae505-4cef-46e5-85cc-cf8b410e654a-kubelet-config\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:36:58.320059 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318568 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-etc-openvswitch\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.320059 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318573 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-run-k8s-cni-cncf-io\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.320059 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318614 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.320059 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318628 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-cni-dir\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.320059 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318651 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-etc-kubernetes\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.320059 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318659 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8a3c8868-93e2-4f20-910e-d2cd73bc1d5f-iptables-alerter-script\") pod \"iptables-alerter-p88j5\" (UID: \"8a3c8868-93e2-4f20-910e-d2cd73bc1d5f\") " pod="openshift-network-operator/iptables-alerter-p88j5" Oct 30 18:36:58.320059 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318680 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.320059 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318687 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-run-k8s-cni-cncf-io\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.320059 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318704 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-etc-kubernetes\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.320059 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318723 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-daemon-config\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.320059 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318750 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a3c8868-93e2-4f20-910e-d2cd73bc1d5f-host-slash\") pod \"iptables-alerter-p88j5\" (UID: \"8a3c8868-93e2-4f20-910e-d2cd73bc1d5f\") " pod="openshift-network-operator/iptables-alerter-p88j5" Oct 30 18:36:58.320059 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318773 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-socket-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.320059 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318795 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftcrx\" (UniqueName: \"kubernetes.io/projected/b7ad340e-8e80-4f55-a29b-e8d501946cb5-kube-api-access-ftcrx\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.320059 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318804 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a3c8868-93e2-4f20-910e-d2cd73bc1d5f-host-slash\") pod \"iptables-alerter-p88j5\" (UID: \"8a3c8868-93e2-4f20-910e-d2cd73bc1d5f\") " pod="openshift-network-operator/iptables-alerter-p88j5" Oct 30 18:36:58.320059 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318819 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-run-netns\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.320059 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318842 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-conf-dir\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.320823 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318866 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-slash\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.320823 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318888 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-var-lib-kubelet\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.320823 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318910 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-cni-binary-copy\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.320823 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318899 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-socket-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.320823 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318936 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d8e44bd9-13d6-4e54-9d9b-bfad33e1393c-hosts-file\") pod \"node-resolver-k7r7n\" (UID: \"d8e44bd9-13d6-4e54-9d9b-bfad33e1393c\") " pod="openshift-dns/node-resolver-k7r7n" Oct 30 18:36:58.320823 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318963 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-run-openvswitch\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.320823 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.318965 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-slash\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.320823 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319018 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-log-socket\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.320823 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319037 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-run-netns\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.320823 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319043 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-run-ovn-kubernetes\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.320823 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319077 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-cni-bin\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.320823 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319095 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-conf-dir\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.320823 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319100 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/70781b52-75b2-450a-a058-b0d609659cda-ovnkube-config\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.320823 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319123 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/70781b52-75b2-450a-a058-b0d609659cda-env-overrides\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.320823 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319137 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/10a7e4f4-2efc-4803-b3c2-da8749031462-cni-binary-copy\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.320823 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319149 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/70781b52-75b2-450a-a058-b0d609659cda-ovn-node-metrics-cert\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.320823 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319155 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-var-lib-kubelet\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.320823 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319171 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a29b358-3cb7-49e6-85a9-6581a2f52bd1-host\") pod \"node-ca-mxjc4\" (UID: \"6a29b358-3cb7-49e6-85a9-6581a2f52bd1\") " pod="openshift-image-registry/node-ca-mxjc4" Oct 30 18:36:58.321463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319193 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-run-ovn-kubernetes\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.321463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319197 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-run-openvswitch\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.321463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319195 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-var-lib-cni-multus\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.321463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319196 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.321463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319228 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-daemon-config\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.321463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319246 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-log-socket\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.321463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319276 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d8e44bd9-13d6-4e54-9d9b-bfad33e1393c-hosts-file\") pod \"node-resolver-k7r7n\" (UID: \"d8e44bd9-13d6-4e54-9d9b-bfad33e1393c\") " pod="openshift-dns/node-resolver-k7r7n" Oct 30 18:36:58.321463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319287 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-cni-bin\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.321463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319246 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-var-lib-cni-multus\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.321463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319323 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-hostroot\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.321463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319348 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-host\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.321463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319372 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-device-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.321463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319396 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-hostroot\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.321463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319331 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a29b358-3cb7-49e6-85a9-6581a2f52bd1-host\") pod \"node-ca-mxjc4\" (UID: \"6a29b358-3cb7-49e6-85a9-6581a2f52bd1\") " pod="openshift-image-registry/node-ca-mxjc4" Oct 30 18:36:58.321463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319414 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-system-cni-dir\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.321463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319440 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5tc8\" (UniqueName: \"kubernetes.io/projected/d8e44bd9-13d6-4e54-9d9b-bfad33e1393c-kube-api-access-l5tc8\") pod \"node-resolver-k7r7n\" (UID: \"d8e44bd9-13d6-4e54-9d9b-bfad33e1393c\") " pod="openshift-dns/node-resolver-k7r7n" Oct 30 18:36:58.321463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319451 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-host\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.321463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319478 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tww25\" (UniqueName: \"kubernetes.io/projected/70781b52-75b2-450a-a058-b0d609659cda-kube-api-access-tww25\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.321949 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319487 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-cni-binary-copy\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.321949 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319492 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-system-cni-dir\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.321949 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319503 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr22p\" (UniqueName: \"kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p\") pod \"network-check-target-62vfn\" (UID: \"97294b18-463b-429d-8073-dc532ab0a046\") " pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:36:58.321949 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319550 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-system-cni-dir\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.321949 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319578 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-systemd\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.321949 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319619 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-tuned\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.321949 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319630 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/70781b52-75b2-450a-a058-b0d609659cda-env-overrides\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.321949 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319644 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-cni-netd\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.321949 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319674 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-systemd\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.321949 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319718 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-cni-netd\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.321949 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319728 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-device-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.321949 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319755 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1bec85c3-72c2-46dc-9f68-8e80f1478422-agent-certs\") pod \"konnectivity-agent-w2wtr\" (UID: \"1bec85c3-72c2-46dc-9f68-8e80f1478422\") " pod="kube-system/konnectivity-agent-w2wtr" Oct 30 18:36:58.321949 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319767 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/70781b52-75b2-450a-a058-b0d609659cda-ovnkube-config\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.321949 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319780 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhpbc\" (UniqueName: \"kubernetes.io/projected/10a7e4f4-2efc-4803-b3c2-da8749031462-kube-api-access-nhpbc\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.321949 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319837 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-var-lib-kubelet\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.321949 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319774 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-system-cni-dir\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.321949 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319881 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-etc-selinux\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.322442 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319915 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-var-lib-kubelet\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.322442 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319929 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-run-netns\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.322442 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319953 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs\") pod \"network-metrics-daemon-phngs\" (UID: \"25bce85b-a92c-4c39-a298-7b1881380069\") " pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:36:58.322442 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319986 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-socket-dir-parent\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.322442 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.319997 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-run-netns\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.322442 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320024 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-sysctl-conf\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.322442 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320017 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-etc-selinux\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.322442 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:58.320089 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:36:58.322442 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320100 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-socket-dir-parent\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.322442 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:58.320135 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs podName:25bce85b-a92c-4c39-a298-7b1881380069 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:58.820120572 +0000 UTC m=+3.130666477 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs") pod "network-metrics-daemon-phngs" (UID: "25bce85b-a92c-4c39-a298-7b1881380069") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:36:58.322442 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320149 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-sysctl-conf\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.322442 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320180 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7n2h\" (UniqueName: \"kubernetes.io/projected/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-kube-api-access-n7n2h\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.322442 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320206 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-node-log\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.322442 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320243 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbb9x\" (UniqueName: \"kubernetes.io/projected/6a29b358-3cb7-49e6-85a9-6581a2f52bd1-kube-api-access-gbb9x\") pod \"node-ca-mxjc4\" (UID: \"6a29b358-3cb7-49e6-85a9-6581a2f52bd1\") " pod="openshift-image-registry/node-ca-mxjc4" Oct 30 18:36:58.322442 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320248 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-node-log\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.322442 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320269 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-kubernetes\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.322442 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320292 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-lib-modules\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.322886 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320316 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-kubelet\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.322886 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320415 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/70781b52-75b2-450a-a058-b0d609659cda-ovnkube-script-lib\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.322886 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320447 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxznv\" (UniqueName: \"kubernetes.io/projected/25bce85b-a92c-4c39-a298-7b1881380069-kube-api-access-fxznv\") pod \"network-metrics-daemon-phngs\" (UID: \"25bce85b-a92c-4c39-a298-7b1881380069\") " pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:36:58.322886 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320463 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-lib-modules\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.322886 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320503 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-kubelet\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.322886 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320541 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1bec85c3-72c2-46dc-9f68-8e80f1478422-konnectivity-ca\") pod \"konnectivity-agent-w2wtr\" (UID: \"1bec85c3-72c2-46dc-9f68-8e80f1478422\") " pod="kube-system/konnectivity-agent-w2wtr" Oct 30 18:36:58.322886 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320570 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-sysconfig\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.322886 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320594 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-sys\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.322886 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320620 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-registration-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.322886 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320660 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-sysconfig\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.322886 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320679 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-sys\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.322886 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320698 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-registration-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.322886 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.320980 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/70781b52-75b2-450a-a058-b0d609659cda-ovnkube-script-lib\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.322886 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.321068 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1bec85c3-72c2-46dc-9f68-8e80f1478422-konnectivity-ca\") pod \"konnectivity-agent-w2wtr\" (UID: \"1bec85c3-72c2-46dc-9f68-8e80f1478422\") " pod="kube-system/konnectivity-agent-w2wtr" Oct 30 18:36:58.322886 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.321081 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-kubernetes\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.322886 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.322220 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-tuned\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.322886 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.322287 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8443eba-410b-4aea-94b7-f5fdabbb06e3-tmp\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.322886 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.322491 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1bec85c3-72c2-46dc-9f68-8e80f1478422-agent-certs\") pod \"konnectivity-agent-w2wtr\" (UID: \"1bec85c3-72c2-46dc-9f68-8e80f1478422\") " pod="kube-system/konnectivity-agent-w2wtr" Oct 30 18:36:58.323361 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.322584 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/70781b52-75b2-450a-a058-b0d609659cda-ovn-node-metrics-cert\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.377204 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.377135 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q64j\" (UniqueName: \"kubernetes.io/projected/d8443eba-410b-4aea-94b7-f5fdabbb06e3-kube-api-access-9q64j\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.377657 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.377639 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8pcp\" (UniqueName: \"kubernetes.io/projected/8a3c8868-93e2-4f20-910e-d2cd73bc1d5f-kube-api-access-v8pcp\") pod \"iptables-alerter-p88j5\" (UID: \"8a3c8868-93e2-4f20-910e-d2cd73bc1d5f\") " pod="openshift-network-operator/iptables-alerter-p88j5" Oct 30 18:36:58.377758 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.377744 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftcrx\" (UniqueName: \"kubernetes.io/projected/b7ad340e-8e80-4f55-a29b-e8d501946cb5-kube-api-access-ftcrx\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.382840 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.382820 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7n2h\" (UniqueName: \"kubernetes.io/projected/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-kube-api-access-n7n2h\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.386927 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:58.386911 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:36:58.387014 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:58.386930 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:36:58.387014 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:58.386939 2581 projected.go:196] Error preparing data for projected volume kube-api-access-tr22p for pod openshift-network-diagnostics/network-check-target-62vfn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:36:58.387122 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:58.387025 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p podName:97294b18-463b-429d-8073-dc532ab0a046 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:58.886992391 +0000 UTC m=+3.197538279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tr22p" (UniqueName: "kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p") pod "network-check-target-62vfn" (UID: "97294b18-463b-429d-8073-dc532ab0a046") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:36:58.388586 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.388563 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxznv\" (UniqueName: \"kubernetes.io/projected/25bce85b-a92c-4c39-a298-7b1881380069-kube-api-access-fxznv\") pod \"network-metrics-daemon-phngs\" (UID: \"25bce85b-a92c-4c39-a298-7b1881380069\") " pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:36:58.388958 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.388930 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tww25\" (UniqueName: \"kubernetes.io/projected/70781b52-75b2-450a-a058-b0d609659cda-kube-api-access-tww25\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.389101 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.389085 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhpbc\" (UniqueName: \"kubernetes.io/projected/10a7e4f4-2efc-4803-b3c2-da8749031462-kube-api-access-nhpbc\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:36:58.389145 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.389087 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5tc8\" (UniqueName: \"kubernetes.io/projected/d8e44bd9-13d6-4e54-9d9b-bfad33e1393c-kube-api-access-l5tc8\") pod \"node-resolver-k7r7n\" (UID: \"d8e44bd9-13d6-4e54-9d9b-bfad33e1393c\") " pod="openshift-dns/node-resolver-k7r7n" Oct 30 18:36:58.393748 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.393732 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbb9x\" (UniqueName: \"kubernetes.io/projected/6a29b358-3cb7-49e6-85a9-6581a2f52bd1-kube-api-access-gbb9x\") pod \"node-ca-mxjc4\" (UID: \"6a29b358-3cb7-49e6-85a9-6581a2f52bd1\") " pod="openshift-image-registry/node-ca-mxjc4" Oct 30 18:36:58.502386 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.502345 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-928bp" Oct 30 18:36:58.504468 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.504450 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w2wtr" Oct 30 18:36:58.512192 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:58.512163 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10a7e4f4_2efc_4803_b3c2_da8749031462.slice/crio-9e306e4abb478f7c63370fec934676c09baf889677a0d28fae31d6be10cb4d60 WatchSource:0}: Error finding container 9e306e4abb478f7c63370fec934676c09baf889677a0d28fae31d6be10cb4d60: Status 404 returned error can't find the container with id 9e306e4abb478f7c63370fec934676c09baf889677a0d28fae31d6be10cb4d60 Oct 30 18:36:58.512815 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.512787 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:36:58.512895 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.512859 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mxjc4" Oct 30 18:36:58.514056 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:58.514031 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bec85c3_72c2_46dc_9f68_8e80f1478422.slice/crio-8e66dd96cdce3c9ee74b663d5a10f651c9b9e31f53a7783994ceb363539002ec WatchSource:0}: Error finding container 8e66dd96cdce3c9ee74b663d5a10f651c9b9e31f53a7783994ceb363539002ec: Status 404 returned error can't find the container with id 8e66dd96cdce3c9ee74b663d5a10f651c9b9e31f53a7783994ceb363539002ec Oct 30 18:36:58.516119 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.516042 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p88j5" Oct 30 18:36:58.521245 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.521230 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:36:58.524075 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:58.524052 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70781b52_75b2_450a_a058_b0d609659cda.slice/crio-b4aab2d08faef690cce824193c9feb51340f503e938738e69988c5bd1f3a1895 WatchSource:0}: Error finding container b4aab2d08faef690cce824193c9feb51340f503e938738e69988c5bd1f3a1895: Status 404 returned error can't find the container with id b4aab2d08faef690cce824193c9feb51340f503e938738e69988c5bd1f3a1895 Oct 30 18:36:58.529337 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:58.529310 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a29b358_3cb7_49e6_85a9_6581a2f52bd1.slice/crio-4a00b3fe16bdd89c160b2ea6b60a3eb41037c475078a4aa24f4a18b7573c00e6 WatchSource:0}: Error finding container 4a00b3fe16bdd89c160b2ea6b60a3eb41037c475078a4aa24f4a18b7573c00e6: Status 404 returned error can't find the container with id 4a00b3fe16bdd89c160b2ea6b60a3eb41037c475078a4aa24f4a18b7573c00e6 Oct 30 18:36:58.529971 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:58.529951 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a3c8868_93e2_4f20_910e_d2cd73bc1d5f.slice/crio-6672e5ae55c20f12333e11f893df445070994a9ed7dd04e8037d93aa419b7b0f WatchSource:0}: Error finding container 6672e5ae55c20f12333e11f893df445070994a9ed7dd04e8037d93aa419b7b0f: Status 404 returned error can't find the container with id 6672e5ae55c20f12333e11f893df445070994a9ed7dd04e8037d93aa419b7b0f Oct 30 18:36:58.531821 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.531807 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:36:58.532636 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:58.532617 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8443eba_410b_4aea_94b7_f5fdabbb06e3.slice/crio-c35b3e72da7b495638b1961fa8c71f8e28d6aacc4fea703724340484e3d2f0f4 WatchSource:0}: Error finding container c35b3e72da7b495638b1961fa8c71f8e28d6aacc4fea703724340484e3d2f0f4: Status 404 returned error can't find the container with id c35b3e72da7b495638b1961fa8c71f8e28d6aacc4fea703724340484e3d2f0f4 Oct 30 18:36:58.540837 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:58.540818 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7ad340e_8e80_4f55_a29b_e8d501946cb5.slice/crio-acde37ead4bf7d34e9527554b79707fc35045e9df95cf19f8cd6e02118aa0949 WatchSource:0}: Error finding container acde37ead4bf7d34e9527554b79707fc35045e9df95cf19f8cd6e02118aa0949: Status 404 returned error can't find the container with id acde37ead4bf7d34e9527554b79707fc35045e9df95cf19f8cd6e02118aa0949 Oct 30 18:36:58.544486 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.544470 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k7r7n" Oct 30 18:36:58.548993 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.548978 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:36:58.553260 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:36:58.553234 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8e44bd9_13d6_4e54_9d9b_bfad33e1393c.slice/crio-1c5f8f6c3b5058226e9e427143606a83a1fdea05c61212411e392526543e80c1 WatchSource:0}: Error finding container 1c5f8f6c3b5058226e9e427143606a83a1fdea05c61212411e392526543e80c1: Status 404 returned error can't find the container with id 1c5f8f6c3b5058226e9e427143606a83a1fdea05c61212411e392526543e80c1 Oct 30 18:36:58.824808 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.824724 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs\") pod \"network-metrics-daemon-phngs\" (UID: \"25bce85b-a92c-4c39-a298-7b1881380069\") " pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:36:58.824808 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.824781 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:36:58.824973 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:58.824902 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:36:58.824973 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:58.824956 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret podName:f1cae505-4cef-46e5-85cc-cf8b410e654a nodeName:}" failed. No retries permitted until 2026-10-30 18:36:59.824939754 +0000 UTC m=+4.135485644 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret") pod "global-pull-secret-syncer-pxndl" (UID: "f1cae505-4cef-46e5-85cc-cf8b410e654a") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:36:58.825097 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:58.825033 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:36:58.825097 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:58.825080 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs podName:25bce85b-a92c-4c39-a298-7b1881380069 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:59.825067758 +0000 UTC m=+4.135613637 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs") pod "network-metrics-daemon-phngs" (UID: "25bce85b-a92c-4c39-a298-7b1881380069") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:36:58.925807 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:58.925769 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr22p\" (UniqueName: \"kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p\") pod \"network-check-target-62vfn\" (UID: \"97294b18-463b-429d-8073-dc532ab0a046\") " pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:36:58.925965 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:58.925933 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:36:58.925965 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:58.925958 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:36:58.926084 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:58.925972 2581 projected.go:196] Error preparing data for projected volume kube-api-access-tr22p for pod openshift-network-diagnostics/network-check-target-62vfn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:36:58.926084 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:58.926036 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p podName:97294b18-463b-429d-8073-dc532ab0a046 nodeName:}" failed. No retries permitted until 2026-10-30 18:36:59.926021864 +0000 UTC m=+4.236567739 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tr22p" (UniqueName: "kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p") pod "network-check-target-62vfn" (UID: "97294b18-463b-429d-8073-dc532ab0a046") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:36:59.192257 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:59.192179 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mxjc4" event={"ID":"6a29b358-3cb7-49e6-85a9-6581a2f52bd1","Type":"ContainerStarted","Data":"4a00b3fe16bdd89c160b2ea6b60a3eb41037c475078a4aa24f4a18b7573c00e6"} Oct 30 18:36:59.193313 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:59.193284 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w2wtr" event={"ID":"1bec85c3-72c2-46dc-9f68-8e80f1478422","Type":"ContainerStarted","Data":"8e66dd96cdce3c9ee74b663d5a10f651c9b9e31f53a7783994ceb363539002ec"} Oct 30 18:36:59.194509 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:59.194458 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-928bp" event={"ID":"10a7e4f4-2efc-4803-b3c2-da8749031462","Type":"ContainerStarted","Data":"9e306e4abb478f7c63370fec934676c09baf889677a0d28fae31d6be10cb4d60"} Oct 30 18:36:59.195780 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:59.195748 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" event={"ID":"b7ad340e-8e80-4f55-a29b-e8d501946cb5","Type":"ContainerStarted","Data":"acde37ead4bf7d34e9527554b79707fc35045e9df95cf19f8cd6e02118aa0949"} Oct 30 18:36:59.196952 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:59.196928 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" event={"ID":"d8443eba-410b-4aea-94b7-f5fdabbb06e3","Type":"ContainerStarted","Data":"c35b3e72da7b495638b1961fa8c71f8e28d6aacc4fea703724340484e3d2f0f4"} Oct 30 18:36:59.198189 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:59.198138 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p88j5" event={"ID":"8a3c8868-93e2-4f20-910e-d2cd73bc1d5f","Type":"ContainerStarted","Data":"6672e5ae55c20f12333e11f893df445070994a9ed7dd04e8037d93aa419b7b0f"} Oct 30 18:36:59.199319 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:59.199296 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" event={"ID":"70781b52-75b2-450a-a058-b0d609659cda","Type":"ContainerStarted","Data":"b4aab2d08faef690cce824193c9feb51340f503e938738e69988c5bd1f3a1895"} Oct 30 18:36:59.200539 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:59.200510 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7dkzl" event={"ID":"66d0cf78-7aa4-40f2-ab29-dd33a5c73979","Type":"ContainerStarted","Data":"0ebfc98f0252b4d4ac1ac940a893beb05eb0a976e9fecdec0573c37adc6f42d4"} Oct 30 18:36:59.201611 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:59.201589 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k7r7n" event={"ID":"d8e44bd9-13d6-4e54-9d9b-bfad33e1393c","Type":"ContainerStarted","Data":"1c5f8f6c3b5058226e9e427143606a83a1fdea05c61212411e392526543e80c1"} Oct 30 18:36:59.283202 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:59.283144 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:31:58 +0000 UTC" deadline="2028-04-27 01:58:45.97737989 +0000 UTC" Oct 30 18:36:59.283202 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:59.283201 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13063h21m46.69418376s" Oct 30 18:36:59.835088 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:59.834683 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs\") pod \"network-metrics-daemon-phngs\" (UID: \"25bce85b-a92c-4c39-a298-7b1881380069\") " pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:36:59.835088 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:59.834737 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:36:59.835088 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:59.834858 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:36:59.835088 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:59.834913 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret podName:f1cae505-4cef-46e5-85cc-cf8b410e654a nodeName:}" failed. No retries permitted until 2026-10-30 18:37:01.834896548 +0000 UTC m=+6.145442430 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret") pod "global-pull-secret-syncer-pxndl" (UID: "f1cae505-4cef-46e5-85cc-cf8b410e654a") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:36:59.835088 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:59.835013 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:36:59.835088 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:59.835054 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs podName:25bce85b-a92c-4c39-a298-7b1881380069 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:01.835040457 +0000 UTC m=+6.145586333 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs") pod "network-metrics-daemon-phngs" (UID: "25bce85b-a92c-4c39-a298-7b1881380069") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:36:59.935733 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:36:59.935698 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr22p\" (UniqueName: \"kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p\") pod \"network-check-target-62vfn\" (UID: \"97294b18-463b-429d-8073-dc532ab0a046\") " pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:36:59.935897 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:59.935862 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:36:59.935897 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:59.935886 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:36:59.936036 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:59.935898 2581 projected.go:196] Error preparing data for projected volume kube-api-access-tr22p for pod openshift-network-diagnostics/network-check-target-62vfn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:36:59.936036 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:36:59.935951 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p podName:97294b18-463b-429d-8073-dc532ab0a046 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:01.935934663 +0000 UTC m=+6.246480552 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tr22p" (UniqueName: "kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p") pod "network-check-target-62vfn" (UID: "97294b18-463b-429d-8073-dc532ab0a046") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:00.184381 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:00.184305 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:00.184517 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:00.184461 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-phngs" podUID="25bce85b-a92c-4c39-a298-7b1881380069" Oct 30 18:37:00.184583 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:00.184542 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:00.184631 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:00.184613 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pxndl" podUID="f1cae505-4cef-46e5-85cc-cf8b410e654a" Oct 30 18:37:00.184684 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:00.184668 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:00.184754 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:00.184729 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62vfn" podUID="97294b18-463b-429d-8073-dc532ab0a046" Oct 30 18:37:00.283909 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:00.283846 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:31:58 +0000 UTC" deadline="2028-06-10 14:15:17.917203436 +0000 UTC" Oct 30 18:37:00.283909 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:00.283906 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14131h38m17.633301697s" Oct 30 18:37:01.216897 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:01.216798 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal" event={"ID":"958b813bafdd4852b0fbfdc2a07cf4ae","Type":"ContainerStarted","Data":"86ae2d7ac5c0b4bab8e6719ca2a39775a183030b6b26100416d3fae09434cd54"} Oct 30 18:37:01.328509 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:01.328045 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal" podStartSLOduration=4.32803075 podStartE2EDuration="4.32803075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:01.327617589 +0000 UTC m=+5.638163486" watchObservedRunningTime="2026-10-30 18:37:01.32803075 +0000 UTC m=+5.638576655" Oct 30 18:37:01.861911 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:01.861670 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs\") pod \"network-metrics-daemon-phngs\" (UID: \"25bce85b-a92c-4c39-a298-7b1881380069\") " pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:01.862191 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:01.861946 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:01.862191 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:01.861874 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:37:01.862191 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:01.862064 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs podName:25bce85b-a92c-4c39-a298-7b1881380069 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:05.862044122 +0000 UTC m=+10.172590010 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs") pod "network-metrics-daemon-phngs" (UID: "25bce85b-a92c-4c39-a298-7b1881380069") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:37:01.862191 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:01.862072 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:37:01.862191 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:01.862123 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret podName:f1cae505-4cef-46e5-85cc-cf8b410e654a nodeName:}" failed. No retries permitted until 2026-10-30 18:37:05.862107654 +0000 UTC m=+10.172653547 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret") pod "global-pull-secret-syncer-pxndl" (UID: "f1cae505-4cef-46e5-85cc-cf8b410e654a") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:37:01.963174 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:01.963140 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr22p\" (UniqueName: \"kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p\") pod \"network-check-target-62vfn\" (UID: \"97294b18-463b-429d-8073-dc532ab0a046\") " pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:01.963346 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:01.963289 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:37:01.963346 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:01.963303 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:37:01.963455 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:01.963356 2581 projected.go:196] Error preparing data for projected volume kube-api-access-tr22p for pod openshift-network-diagnostics/network-check-target-62vfn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:01.963455 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:01.963422 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p podName:97294b18-463b-429d-8073-dc532ab0a046 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:05.963405244 +0000 UTC m=+10.273951130 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tr22p" (UniqueName: "kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p") pod "network-check-target-62vfn" (UID: "97294b18-463b-429d-8073-dc532ab0a046") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:02.192873 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:02.192608 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:02.193549 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:02.193289 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pxndl" podUID="f1cae505-4cef-46e5-85cc-cf8b410e654a" Oct 30 18:37:02.193549 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:02.192845 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:02.193549 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:02.193438 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-phngs" podUID="25bce85b-a92c-4c39-a298-7b1881380069" Oct 30 18:37:02.193549 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:02.192786 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:02.193549 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:02.193514 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62vfn" podUID="97294b18-463b-429d-8073-dc532ab0a046" Oct 30 18:37:02.242517 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:02.242483 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7dkzl" event={"ID":"66d0cf78-7aa4-40f2-ab29-dd33a5c73979","Type":"ContainerStarted","Data":"15b9618f25cc8ee09487bc989b2e2c920d504c3673120dd66d63c767287e79a4"} Oct 30 18:37:03.248930 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:03.248176 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w2wtr" event={"ID":"1bec85c3-72c2-46dc-9f68-8e80f1478422","Type":"ContainerStarted","Data":"b27e27157d7c52bd172f38092f7180ecc74b37718c71f69124493c59cf98a1f0"} Oct 30 18:37:03.253161 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:03.252562 2581 generic.go:356] "Generic (PLEG): container finished" podID="6a99a11e235547936507bfb99804f96f" containerID="d90db88c72aeee52c7b6465eff8b29333b67359fdf62d6a008174f4357dfb5bd" exitCode=0 Oct 30 18:37:03.253161 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:03.252641 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" event={"ID":"6a99a11e235547936507bfb99804f96f","Type":"ContainerDied","Data":"d90db88c72aeee52c7b6465eff8b29333b67359fdf62d6a008174f4357dfb5bd"} Oct 30 18:37:03.258506 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:03.258468 2581 generic.go:356] "Generic (PLEG): container finished" podID="66d0cf78-7aa4-40f2-ab29-dd33a5c73979" containerID="15b9618f25cc8ee09487bc989b2e2c920d504c3673120dd66d63c767287e79a4" exitCode=0 Oct 30 18:37:03.258506 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:03.258503 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7dkzl" event={"ID":"66d0cf78-7aa4-40f2-ab29-dd33a5c73979","Type":"ContainerDied","Data":"15b9618f25cc8ee09487bc989b2e2c920d504c3673120dd66d63c767287e79a4"} Oct 30 18:37:03.359302 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:03.359244 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-w2wtr" podStartSLOduration=3.208491529 podStartE2EDuration="7.359225363s" totalImagesPullingTime="4.150733834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:03.358303453 +0000 UTC m=+7.668849353" watchObservedRunningTime="2026-10-30 18:37:03.359225363 +0000 UTC m=+7.669771263" Oct 30 18:37:04.181794 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:04.181759 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:04.181967 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:04.181875 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62vfn" podUID="97294b18-463b-429d-8073-dc532ab0a046" Oct 30 18:37:04.181967 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:04.181949 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:04.182121 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:04.182054 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-phngs" podUID="25bce85b-a92c-4c39-a298-7b1881380069" Oct 30 18:37:04.182121 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:04.182116 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:04.182224 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:04.182190 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pxndl" podUID="f1cae505-4cef-46e5-85cc-cf8b410e654a" Oct 30 18:37:04.265677 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:04.265642 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" event={"ID":"6a99a11e235547936507bfb99804f96f","Type":"ContainerStarted","Data":"272e8b415a40600a7873b9e65a4bbd3ade2469e9b210b2ace6dd1701765841ee"} Oct 30 18:37:04.376584 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:04.376541 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" podStartSLOduration=7.376527568 podStartE2EDuration="7.376527568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:04.376205545 +0000 UTC m=+8.686751443" watchObservedRunningTime="2026-10-30 18:37:04.376527568 +0000 UTC m=+8.687073465" Oct 30 18:37:04.486882 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:04.486837 2581 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-w2wtr" Oct 30 18:37:04.487606 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:04.487587 2581 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-w2wtr" Oct 30 18:37:05.269820 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:05.269781 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mxjc4" event={"ID":"6a29b358-3cb7-49e6-85a9-6581a2f52bd1","Type":"ContainerStarted","Data":"1119754829da8c48b8b958e9322ed3621837e37e3291dd504a3557cae2c1f783"} Oct 30 18:37:05.274099 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:05.274047 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" event={"ID":"b7ad340e-8e80-4f55-a29b-e8d501946cb5","Type":"ContainerStarted","Data":"b72537d0ebe71a60f06fdc0c1f71d9859c6ab98776649fd5b2a3297233203fb2"} Oct 30 18:37:05.382900 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:05.382354 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mxjc4" podStartSLOduration=3.598422931 podStartE2EDuration="9.382334677s" totalImagesPullingTime="5.783911746s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:05.382168189 +0000 UTC m=+9.692714109" watchObservedRunningTime="2026-10-30 18:37:05.382334677 +0000 UTC m=+9.692880576" Oct 30 18:37:05.894957 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:05.894242 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs\") pod \"network-metrics-daemon-phngs\" (UID: \"25bce85b-a92c-4c39-a298-7b1881380069\") " pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:05.894957 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:05.894302 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:05.894957 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:05.894424 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:37:05.894957 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:05.894482 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret podName:f1cae505-4cef-46e5-85cc-cf8b410e654a nodeName:}" failed. No retries permitted until 2026-10-30 18:37:13.894464973 +0000 UTC m=+18.205010873 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret") pod "global-pull-secret-syncer-pxndl" (UID: "f1cae505-4cef-46e5-85cc-cf8b410e654a") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:37:05.894957 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:05.894880 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:37:05.894957 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:05.894922 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs podName:25bce85b-a92c-4c39-a298-7b1881380069 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:13.894911343 +0000 UTC m=+18.205457219 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs") pod "network-metrics-daemon-phngs" (UID: "25bce85b-a92c-4c39-a298-7b1881380069") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:37:05.995212 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:05.994919 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr22p\" (UniqueName: \"kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p\") pod \"network-check-target-62vfn\" (UID: \"97294b18-463b-429d-8073-dc532ab0a046\") " pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:05.995212 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:05.995095 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:37:05.995212 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:05.995117 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:37:05.995212 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:05.995129 2581 projected.go:196] Error preparing data for projected volume kube-api-access-tr22p for pod openshift-network-diagnostics/network-check-target-62vfn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:05.995212 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:05.995176 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p podName:97294b18-463b-429d-8073-dc532ab0a046 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:13.995160216 +0000 UTC m=+18.305706110 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tr22p" (UniqueName: "kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p") pod "network-check-target-62vfn" (UID: "97294b18-463b-429d-8073-dc532ab0a046") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:06.184051 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:06.183960 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:06.184210 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:06.184123 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-phngs" podUID="25bce85b-a92c-4c39-a298-7b1881380069" Oct 30 18:37:06.184545 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:06.184525 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:06.184663 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:06.184617 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pxndl" podUID="f1cae505-4cef-46e5-85cc-cf8b410e654a" Oct 30 18:37:06.184725 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:06.184673 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:06.184779 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:06.184738 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62vfn" podUID="97294b18-463b-429d-8073-dc532ab0a046" Oct 30 18:37:06.276441 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:06.276418 2581 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:37:06.677167 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:06.676882 2581 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 18:37:07.162059 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:07.161375 2581 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:37:06.676899915Z","UUID":"d0746bdb-fe7c-4c43-830b-f445649e641b","Handler":null,"Name":"","Endpoint":""} Oct 30 18:37:07.163327 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:07.163304 2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 18:37:07.163452 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:07.163337 2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 18:37:07.281670 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:07.281636 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-928bp" event={"ID":"10a7e4f4-2efc-4803-b3c2-da8749031462","Type":"ContainerStarted","Data":"89a8cf74437e75907e66be84e5d5dbc6dbe413ce6dcf0baddd6ee00c4d03c2af"} Oct 30 18:37:07.285984 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:07.285901 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" event={"ID":"b7ad340e-8e80-4f55-a29b-e8d501946cb5","Type":"ContainerStarted","Data":"0d1928eb3e749d6c985de5d7521da34015ec6ed58bd0ad7a149205cbf5337f8b"} Oct 30 18:37:07.397318 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:07.397259 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-928bp" podStartSLOduration=3.3349440599999998 podStartE2EDuration="11.397242955s" totalImagesPullingTime="8.062298895s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:07.39667102 +0000 UTC m=+11.707216920" watchObservedRunningTime="2026-10-30 18:37:07.397242955 +0000 UTC m=+11.707788854" Oct 30 18:37:08.182854 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:08.182633 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:08.183033 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:08.182959 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pxndl" podUID="f1cae505-4cef-46e5-85cc-cf8b410e654a" Oct 30 18:37:08.183165 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:08.183142 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:08.183246 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:08.183226 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62vfn" podUID="97294b18-463b-429d-8073-dc532ab0a046" Oct 30 18:37:08.183306 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:08.183293 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:08.183390 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:08.183374 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-phngs" podUID="25bce85b-a92c-4c39-a298-7b1881380069" Oct 30 18:37:08.291037 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:08.290988 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" event={"ID":"b7ad340e-8e80-4f55-a29b-e8d501946cb5","Type":"ContainerStarted","Data":"3d75cab30b29dd7130e0431f71cfa02c79f62830365552ae4b9588f9e77b0d05"} Oct 30 18:37:08.293971 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:08.293924 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p88j5" event={"ID":"8a3c8868-93e2-4f20-910e-d2cd73bc1d5f","Type":"ContainerStarted","Data":"5cda87e289e899b481594e1c3d60d9090507dbd5a63e734752f803faa2a3268b"} Oct 30 18:37:08.296104 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:08.296016 2581 generic.go:356] "Generic (PLEG): container finished" podID="66d0cf78-7aa4-40f2-ab29-dd33a5c73979" containerID="16772c9016af1b2d147ba8ac46d0ef276b8d1dccca6abd60fdccd6546392300b" exitCode=0 Oct 30 18:37:08.296227 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:08.296099 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7dkzl" event={"ID":"66d0cf78-7aa4-40f2-ab29-dd33a5c73979","Type":"ContainerDied","Data":"16772c9016af1b2d147ba8ac46d0ef276b8d1dccca6abd60fdccd6546392300b"} Oct 30 18:37:08.299550 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:08.299527 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k7r7n" event={"ID":"d8e44bd9-13d6-4e54-9d9b-bfad33e1393c","Type":"ContainerStarted","Data":"0408a425785c79960f1272bf41e1908fdeb93b1bf1c6a0c5a022630383c4cb78"} Oct 30 18:37:08.405073 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:08.404972 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" podStartSLOduration=3.180211268 podStartE2EDuration="12.404959274s" totalImagesPullingTime="9.224748006s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:08.404382999 +0000 UTC m=+12.714928895" watchObservedRunningTime="2026-10-30 18:37:08.404959274 +0000 UTC m=+12.715505171" Oct 30 18:37:08.641223 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:08.641171 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-p88j5" podStartSLOduration=4.446207745 podStartE2EDuration="12.641155847s" totalImagesPullingTime="8.194948102s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:08.640818364 +0000 UTC m=+12.951364262" watchObservedRunningTime="2026-10-30 18:37:08.641155847 +0000 UTC m=+12.951701742" Oct 30 18:37:09.303067 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:09.303030 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" event={"ID":"d8443eba-410b-4aea-94b7-f5fdabbb06e3","Type":"ContainerStarted","Data":"c62d551bdd08fcfeb52e8f38071585558783b62c823c5a37ae3ee30cabd99e99"} Oct 30 18:37:09.416291 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:09.416239 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-k7r7n" podStartSLOduration=4.627707386 podStartE2EDuration="13.416220182s" totalImagesPullingTime="8.788512796s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:08.767286588 +0000 UTC m=+13.077832483" watchObservedRunningTime="2026-10-30 18:37:09.416220182 +0000 UTC m=+13.726766082" Oct 30 18:37:09.416518 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:09.416480 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" podStartSLOduration=2.7520758130000003 podStartE2EDuration="13.416470411s" totalImagesPullingTime="10.664394598s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:09.415903873 +0000 UTC m=+13.726449773" watchObservedRunningTime="2026-10-30 18:37:09.416470411 +0000 UTC m=+13.727016310" Oct 30 18:37:10.182190 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:10.182151 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:10.182190 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:10.182186 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:10.182397 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:10.182151 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:10.182397 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:10.182283 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-phngs" podUID="25bce85b-a92c-4c39-a298-7b1881380069" Oct 30 18:37:10.182397 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:10.182366 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pxndl" podUID="f1cae505-4cef-46e5-85cc-cf8b410e654a" Oct 30 18:37:10.182539 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:10.182450 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62vfn" podUID="97294b18-463b-429d-8073-dc532ab0a046" Oct 30 18:37:10.306879 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:10.306838 2581 generic.go:356] "Generic (PLEG): container finished" podID="66d0cf78-7aa4-40f2-ab29-dd33a5c73979" containerID="c308cdf08f3f821ca679a1ff83ca8e7fe22c93698f58df805d06708b9ea415cf" exitCode=0 Oct 30 18:37:10.307423 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:10.306929 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7dkzl" event={"ID":"66d0cf78-7aa4-40f2-ab29-dd33a5c73979","Type":"ContainerDied","Data":"c308cdf08f3f821ca679a1ff83ca8e7fe22c93698f58df805d06708b9ea415cf"} Oct 30 18:37:12.182271 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:12.182075 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:12.182842 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:12.182089 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:12.182842 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:12.182367 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pxndl" podUID="f1cae505-4cef-46e5-85cc-cf8b410e654a" Oct 30 18:37:12.182842 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:12.182455 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-phngs" podUID="25bce85b-a92c-4c39-a298-7b1881380069" Oct 30 18:37:12.182842 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:12.182153 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:12.182842 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:12.182542 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62vfn" podUID="97294b18-463b-429d-8073-dc532ab0a046" Oct 30 18:37:12.208133 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:12.208109 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-w2wtr" Oct 30 18:37:12.208296 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:12.208239 2581 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:37:12.209221 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:12.209056 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-w2wtr" Oct 30 18:37:12.314306 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:12.314273 2581 generic.go:356] "Generic (PLEG): container finished" podID="66d0cf78-7aa4-40f2-ab29-dd33a5c73979" containerID="b99c31f6cba7810ab9c68a7e292526361b870a082a3b0256717db2149c7a32b5" exitCode=0 Oct 30 18:37:12.314477 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:12.314359 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7dkzl" event={"ID":"66d0cf78-7aa4-40f2-ab29-dd33a5c73979","Type":"ContainerDied","Data":"b99c31f6cba7810ab9c68a7e292526361b870a082a3b0256717db2149c7a32b5"} Oct 30 18:37:13.959243 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:13.959071 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs\") pod \"network-metrics-daemon-phngs\" (UID: \"25bce85b-a92c-4c39-a298-7b1881380069\") " pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:13.959595 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:13.959277 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:13.959595 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:13.959220 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:37:13.959595 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:13.959387 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs podName:25bce85b-a92c-4c39-a298-7b1881380069 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:29.959366678 +0000 UTC m=+34.269912571 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs") pod "network-metrics-daemon-phngs" (UID: "25bce85b-a92c-4c39-a298-7b1881380069") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:37:13.959595 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:13.959427 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:37:13.959595 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:13.959476 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret podName:f1cae505-4cef-46e5-85cc-cf8b410e654a nodeName:}" failed. No retries permitted until 2026-10-30 18:37:29.959462394 +0000 UTC m=+34.270008278 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret") pod "global-pull-secret-syncer-pxndl" (UID: "f1cae505-4cef-46e5-85cc-cf8b410e654a") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:37:14.059945 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:14.059921 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr22p\" (UniqueName: \"kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p\") pod \"network-check-target-62vfn\" (UID: \"97294b18-463b-429d-8073-dc532ab0a046\") " pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:14.060114 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:14.060097 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:37:14.060175 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:14.060121 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:37:14.060175 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:14.060130 2581 projected.go:196] Error preparing data for projected volume kube-api-access-tr22p for pod openshift-network-diagnostics/network-check-target-62vfn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:14.060249 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:14.060174 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p podName:97294b18-463b-429d-8073-dc532ab0a046 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:30.060162135 +0000 UTC m=+34.370708023 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tr22p" (UniqueName: "kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p") pod "network-check-target-62vfn" (UID: "97294b18-463b-429d-8073-dc532ab0a046") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:14.181966 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:14.181930 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:14.182122 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:14.181992 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:14.182122 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:14.181989 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:14.182200 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:14.182116 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pxndl" podUID="f1cae505-4cef-46e5-85cc-cf8b410e654a" Oct 30 18:37:14.182237 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:14.182210 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62vfn" podUID="97294b18-463b-429d-8073-dc532ab0a046" Oct 30 18:37:14.182317 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:14.182287 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-phngs" podUID="25bce85b-a92c-4c39-a298-7b1881380069" Oct 30 18:37:14.321751 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:14.321719 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" event={"ID":"70781b52-75b2-450a-a058-b0d609659cda","Type":"ContainerStarted","Data":"0d19e9344b0940d5110b4f8fac706472a125afe91bed621ade9d6d294d168acd"} Oct 30 18:37:14.321850 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:14.321805 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" event={"ID":"70781b52-75b2-450a-a058-b0d609659cda","Type":"ContainerStarted","Data":"8cdd109cdbcc4160ba8cb284424cba4e0c3d01bd717fdb338caee73582a7d0c3"} Oct 30 18:37:14.321916 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:14.321853 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" event={"ID":"70781b52-75b2-450a-a058-b0d609659cda","Type":"ContainerStarted","Data":"006e8f4e158ce693490b772d9db55f52ca85b621f3a03418e20c9a4bbf854c03"} Oct 30 18:37:14.321967 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:14.321912 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" event={"ID":"70781b52-75b2-450a-a058-b0d609659cda","Type":"ContainerStarted","Data":"8fde355ea6c7d3be5f6b88e851141fbcec44f5776778b9209a6b07d1b9f8daae"} Oct 30 18:37:14.321967 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:14.321961 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" event={"ID":"70781b52-75b2-450a-a058-b0d609659cda","Type":"ContainerStarted","Data":"a506a8717add65c612e34b3a323dfdf01fbd16f8f128c97156b0725e33be06d4"} Oct 30 18:37:14.322084 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:14.322035 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" event={"ID":"70781b52-75b2-450a-a058-b0d609659cda","Type":"ContainerStarted","Data":"5fd5826440890e117042a5fd7ba4de587bdf47970c7b6b408ead2c3c1ddbb8c9"} Oct 30 18:37:15.326022 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:15.325978 2581 generic.go:356] "Generic (PLEG): container finished" podID="66d0cf78-7aa4-40f2-ab29-dd33a5c73979" containerID="abdb22ff8a3c4899daad63c816c67c33a147e7ec76a392d52f46859033d546de" exitCode=0 Oct 30 18:37:15.326022 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:15.326028 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7dkzl" event={"ID":"66d0cf78-7aa4-40f2-ab29-dd33a5c73979","Type":"ContainerDied","Data":"abdb22ff8a3c4899daad63c816c67c33a147e7ec76a392d52f46859033d546de"} Oct 30 18:37:16.182482 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:16.182449 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:16.182585 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:16.182547 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-phngs" podUID="25bce85b-a92c-4c39-a298-7b1881380069" Oct 30 18:37:16.182652 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:16.182597 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:16.182704 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:16.182646 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pxndl" podUID="f1cae505-4cef-46e5-85cc-cf8b410e654a" Oct 30 18:37:16.182704 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:16.182689 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:16.182799 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:16.182761 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62vfn" podUID="97294b18-463b-429d-8073-dc532ab0a046" Oct 30 18:37:16.330746 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:16.330708 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" event={"ID":"70781b52-75b2-450a-a058-b0d609659cda","Type":"ContainerStarted","Data":"402b4c5ab99566de70107ae1e308a61e1a4a4ae1d9526aaddb9786b7d3140271"} Oct 30 18:37:16.333424 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:16.333396 2581 generic.go:356] "Generic (PLEG): container finished" podID="66d0cf78-7aa4-40f2-ab29-dd33a5c73979" containerID="b0286eb871d8e27520f053cfc4524dc67767b66b0848bb534a8a1be04d9d0edc" exitCode=0 Oct 30 18:37:16.333543 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:16.333426 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7dkzl" event={"ID":"66d0cf78-7aa4-40f2-ab29-dd33a5c73979","Type":"ContainerDied","Data":"b0286eb871d8e27520f053cfc4524dc67767b66b0848bb534a8a1be04d9d0edc"} Oct 30 18:37:17.338747 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:17.338711 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7dkzl" event={"ID":"66d0cf78-7aa4-40f2-ab29-dd33a5c73979","Type":"ContainerStarted","Data":"e42f3299518f0f602b18f3888c31675763172e9dabc3c5b8cca834b0eb20528f"} Oct 30 18:37:18.182500 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:18.182465 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:18.182500 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:18.182485 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:18.182680 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:18.182553 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:18.182680 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:18.182568 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pxndl" podUID="f1cae505-4cef-46e5-85cc-cf8b410e654a" Oct 30 18:37:18.182680 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:18.182637 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-phngs" podUID="25bce85b-a92c-4c39-a298-7b1881380069" Oct 30 18:37:18.182813 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:18.182703 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62vfn" podUID="97294b18-463b-429d-8073-dc532ab0a046" Oct 30 18:37:18.345866 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:18.345830 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" event={"ID":"70781b52-75b2-450a-a058-b0d609659cda","Type":"ContainerStarted","Data":"3c1e4bf9238a1e0fd4b8b9bdddc1582a32d67073542b1cc5795de64a245eb1fd"} Oct 30 18:37:18.594206 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:18.594117 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7dkzl" podStartSLOduration=10.977947656 podStartE2EDuration="22.594102595s" totalImagesPullingTime="11.616154939s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:56 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:17.548680001 +0000 UTC m=+21.859225899" watchObservedRunningTime="2026-10-30 18:37:18.594102595 +0000 UTC m=+22.904648493" Oct 30 18:37:18.594345 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:18.594256 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" podStartSLOduration=7.388400242 podStartE2EDuration="22.594249088s" totalImagesPullingTime="15.205848846s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:18.561481787 +0000 UTC m=+22.872027683" watchObservedRunningTime="2026-10-30 18:37:18.594249088 +0000 UTC m=+22.904794986" Oct 30 18:37:19.348378 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:19.348352 2581 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:37:19.348745 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:19.348720 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:19.348833 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:19.348819 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:19.363738 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:19.363717 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:19.363844 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:19.363825 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:19.641123 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:19.641089 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:20.184951 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:20.184774 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:20.185198 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:20.184858 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:20.185198 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:20.185038 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pxndl" podUID="f1cae505-4cef-46e5-85cc-cf8b410e654a" Oct 30 18:37:20.185198 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:20.184865 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:20.185198 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:20.185111 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-phngs" podUID="25bce85b-a92c-4c39-a298-7b1881380069" Oct 30 18:37:20.185198 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:20.185156 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62vfn" podUID="97294b18-463b-429d-8073-dc532ab0a046" Oct 30 18:37:20.891961 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:20.891927 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pxndl"] Oct 30 18:37:20.892420 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:20.892062 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:20.892420 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:20.892180 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pxndl" podUID="f1cae505-4cef-46e5-85cc-cf8b410e654a" Oct 30 18:37:20.897380 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:20.897349 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-62vfn"] Oct 30 18:37:20.897489 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:20.897439 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:20.897553 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:20.897533 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62vfn" podUID="97294b18-463b-429d-8073-dc532ab0a046" Oct 30 18:37:20.900441 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:20.900419 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-phngs"] Oct 30 18:37:20.900516 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:20.900495 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:20.900580 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:20.900565 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-phngs" podUID="25bce85b-a92c-4c39-a298-7b1881380069" Oct 30 18:37:22.182176 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:22.182146 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:22.182528 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:22.182246 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-phngs" podUID="25bce85b-a92c-4c39-a298-7b1881380069" Oct 30 18:37:22.182528 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:22.182323 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:22.182528 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:22.182400 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62vfn" podUID="97294b18-463b-429d-8073-dc532ab0a046" Oct 30 18:37:23.182562 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:23.182476 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:23.182900 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:23.182581 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pxndl" podUID="f1cae505-4cef-46e5-85cc-cf8b410e654a" Oct 30 18:37:24.181725 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:24.181692 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:24.181725 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:24.181719 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:24.182115 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:24.181789 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62vfn" podUID="97294b18-463b-429d-8073-dc532ab0a046" Oct 30 18:37:24.182115 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:24.181910 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-phngs" podUID="25bce85b-a92c-4c39-a298-7b1881380069" Oct 30 18:37:25.181862 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:25.181831 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:25.182228 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:25.181930 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pxndl" podUID="f1cae505-4cef-46e5-85cc-cf8b410e654a" Oct 30 18:37:26.182844 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.182813 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:26.183217 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:26.182900 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-phngs" podUID="25bce85b-a92c-4c39-a298-7b1881380069" Oct 30 18:37:26.183217 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.182978 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:26.183217 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:26.183075 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-62vfn" podUID="97294b18-463b-429d-8073-dc532ab0a046" Oct 30 18:37:26.476792 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.476723 2581 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeReady" Oct 30 18:37:26.476941 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.476849 2581 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Oct 30 18:37:26.627501 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.627470 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-k4dpj"] Oct 30 18:37:26.631430 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.631415 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-k4dpj" Oct 30 18:37:26.632830 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.632808 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-mzqcg"] Oct 30 18:37:26.636315 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.636298 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-mzqcg" Oct 30 18:37:26.664220 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.664199 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq"] Oct 30 18:37:26.667490 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.667474 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-kbclw"] Oct 30 18:37:26.667613 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.667593 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq" Oct 30 18:37:26.670721 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.670701 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-dl96p"] Oct 30 18:37:26.670817 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.670799 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-kbclw" Oct 30 18:37:26.673986 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.673968 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-k4dpj"] Oct 30 18:37:26.674154 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.674133 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-mzqcg"] Oct 30 18:37:26.674241 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.674069 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" Oct 30 18:37:26.692845 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.692830 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-q9rgp"] Oct 30 18:37:26.695745 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.695727 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:26.701551 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.701534 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\"" Oct 30 18:37:26.702180 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.702166 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\"" Oct 30 18:37:26.710810 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.710792 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-649485556b-7x5gc"] Oct 30 18:37:26.714068 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.714052 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ltkzx"] Oct 30 18:37:26.714181 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.714168 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:26.717089 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.717036 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt"] Oct 30 18:37:26.717246 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.717226 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:26.720108 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.720090 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6c975679c5-x5lg8"] Oct 30 18:37:26.720202 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.720187 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:26.723425 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.723409 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:26.724122 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.724104 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b"] Oct 30 18:37:26.725061 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.725042 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Oct 30 18:37:26.727299 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.727223 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-nprgd"] Oct 30 18:37:26.727369 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.727324 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:26.730223 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.730208 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6"] Oct 30 18:37:26.730329 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.730315 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-nprgd" Oct 30 18:37:26.732983 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.732963 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6" Oct 30 18:37:26.739831 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.739312 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-swfq5\"" Oct 30 18:37:26.740642 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.740608 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-tp7sc"] Oct 30 18:37:26.744472 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.744451 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:26.746629 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.746605 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1340fb2d-5f77-4316-a156-f82e0cb1124d-nginx-conf\") pod \"networking-console-plugin-86c478bb88-dl96p\" (UID: \"1340fb2d-5f77-4316-a156-f82e0cb1124d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" Oct 30 18:37:26.746736 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.746645 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/13bb697e-8bff-4a30-9e1d-b9f3baac416e-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-gktbq\" (UID: \"13bb697e-8bff-4a30-9e1d-b9f3baac416e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq" Oct 30 18:37:26.746736 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.746705 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lnmb\" (UniqueName: \"kubernetes.io/projected/14b832bf-1a7b-4539-811d-84a11d2b55b6-kube-api-access-9lnmb\") pod \"kas-connection-checker-685895c7c9-k4dpj\" (UID: \"14b832bf-1a7b-4539-811d-84a11d2b55b6\") " pod="kube-system/kas-connection-checker-685895c7c9-k4dpj" Oct 30 18:37:26.746850 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.746738 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5glk7\" (UniqueName: \"kubernetes.io/projected/13bb697e-8bff-4a30-9e1d-b9f3baac416e-kube-api-access-5glk7\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-gktbq\" (UID: \"13bb697e-8bff-4a30-9e1d-b9f3baac416e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq" Oct 30 18:37:26.746850 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.746831 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8j7j\" (UniqueName: \"kubernetes.io/projected/6050488d-f6ab-4bab-b2b2-90d30f64bab4-kube-api-access-d8j7j\") pod \"kas-connection-checker-685895c7c9-mzqcg\" (UID: \"6050488d-f6ab-4bab-b2b2-90d30f64bab4\") " pod="kube-system/kas-connection-checker-685895c7c9-mzqcg" Oct 30 18:37:26.746988 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.746863 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-dl96p\" (UID: \"1340fb2d-5f77-4316-a156-f82e0cb1124d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" Oct 30 18:37:26.746988 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.746884 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/13bb697e-8bff-4a30-9e1d-b9f3baac416e-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-gktbq\" (UID: \"13bb697e-8bff-4a30-9e1d-b9f3baac416e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq" Oct 30 18:37:26.747107 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.746988 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twdrz\" (UniqueName: \"kubernetes.io/projected/9b5d4a9f-0410-49ba-ab07-c0c69e3fe45f-kube-api-access-twdrz\") pod \"kas-connection-checker-685895c7c9-kbclw\" (UID: \"9b5d4a9f-0410-49ba-ab07-c0c69e3fe45f\") " pod="kube-system/kas-connection-checker-685895c7c9-kbclw" Oct 30 18:37:26.747486 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.747467 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5df4b54dc9-8pdtm"] Oct 30 18:37:26.750270 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.750252 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-dl96p"] Oct 30 18:37:26.750354 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.750282 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-d9fjm"] Oct 30 18:37:26.750433 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.750417 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:26.752982 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.752968 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq"] Oct 30 18:37:26.753078 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.752990 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rbmpn"] Oct 30 18:37:26.753078 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.753069 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-d9fjm" Oct 30 18:37:26.755721 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.755703 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-kbclw"] Oct 30 18:37:26.755796 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.755788 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rbmpn" Oct 30 18:37:26.781695 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.781675 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt"] Oct 30 18:37:26.786662 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.786159 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ltkzx"] Oct 30 18:37:26.787198 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.787178 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6c975679c5-x5lg8"] Oct 30 18:37:26.791759 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.791737 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-tp7sc"] Oct 30 18:37:26.794277 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.794248 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b"] Oct 30 18:37:26.809327 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.809307 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:37:26.809423 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.809309 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Oct 30 18:37:26.809423 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.809362 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 18:37:26.809501 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.809436 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Oct 30 18:37:26.809544 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.809516 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Oct 30 18:37:26.809588 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.809567 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Oct 30 18:37:26.809639 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.809595 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 18:37:26.809696 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.809640 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 18:37:26.809759 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.809747 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 18:37:26.809759 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.809752 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Oct 30 18:37:26.809840 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.809778 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Oct 30 18:37:26.819059 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.819044 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Oct 30 18:37:26.819397 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.819374 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 18:37:26.837733 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.837710 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Oct 30 18:37:26.847457 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.847425 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-config-volume\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:26.847637 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.847469 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lnmb\" (UniqueName: \"kubernetes.io/projected/14b832bf-1a7b-4539-811d-84a11d2b55b6-kube-api-access-9lnmb\") pod \"kas-connection-checker-685895c7c9-k4dpj\" (UID: \"14b832bf-1a7b-4539-811d-84a11d2b55b6\") " pod="kube-system/kas-connection-checker-685895c7c9-k4dpj" Oct 30 18:37:26.847637 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.847495 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-metrics-certs\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:26.847637 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.847524 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/00d3be28-5482-48ea-a1be-629e8941236e-serving-cert\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:26.847637 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.847609 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-image-registry-private-configuration\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:26.847856 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.847640 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-trusted-ca\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:26.847856 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.847660 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w86l\" (UniqueName: \"kubernetes.io/projected/74153c90-cd72-43b8-85c5-aadb4d66a3e4-kube-api-access-8w86l\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:26.847856 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.847676 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:26.847856 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.847703 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw7kf\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-kube-api-access-dw7kf\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:26.847856 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.847753 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0c876fc3-6980-4a99-aed5-7439ca3d6b69-config\") pod \"console-operator-7cd56b8dbc-tp7sc\" (UID: \"0c876fc3-6980-4a99-aed5-7439ca3d6b69\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:26.847856 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.847782 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-trusted-ca\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:26.848159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.847889 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8j7j\" (UniqueName: \"kubernetes.io/projected/6050488d-f6ab-4bab-b2b2-90d30f64bab4-kube-api-access-d8j7j\") pod \"kas-connection-checker-685895c7c9-mzqcg\" (UID: \"6050488d-f6ab-4bab-b2b2-90d30f64bab4\") " pod="kube-system/kas-connection-checker-685895c7c9-mzqcg" Oct 30 18:37:26.848159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.847988 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0c876fc3-6980-4a99-aed5-7439ca3d6b69-serving-cert\") pod \"console-operator-7cd56b8dbc-tp7sc\" (UID: \"0c876fc3-6980-4a99-aed5-7439ca3d6b69\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:26.848159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.848036 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/542d052a-724f-4d98-9be7-7765a8c1d80a-cert\") pod \"ingress-canary-rbmpn\" (UID: \"542d052a-724f-4d98-9be7-7765a8c1d80a\") " pod="openshift-ingress-canary/ingress-canary-rbmpn" Oct 30 18:37:26.848159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.848065 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-certificates\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:26.848159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.848092 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zf4n\" (UniqueName: \"kubernetes.io/projected/b18682b8-f91a-4c27-87eb-82720ba4257f-kube-api-access-5zf4n\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:26.848159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.848117 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-bound-sa-token\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:26.848159 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.848140 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjf45\" (UniqueName: \"kubernetes.io/projected/68788658-4afb-4757-acfa-7638cfb2488b-kube-api-access-mjf45\") pod \"network-check-source-6f78d59f99-d9fjm\" (UID: \"68788658-4afb-4757-acfa-7638cfb2488b\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-d9fjm" Oct 30 18:37:26.848455 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.848178 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-dl96p\" (UID: \"1340fb2d-5f77-4316-a156-f82e0cb1124d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" Oct 30 18:37:26.848455 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.848244 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/13bb697e-8bff-4a30-9e1d-b9f3baac416e-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-gktbq\" (UID: \"13bb697e-8bff-4a30-9e1d-b9f3baac416e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq" Oct 30 18:37:26.848455 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.848307 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:26.848608 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.848471 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twdrz\" (UniqueName: \"kubernetes.io/projected/9b5d4a9f-0410-49ba-ab07-c0c69e3fe45f-kube-api-access-twdrz\") pod \"kas-connection-checker-685895c7c9-kbclw\" (UID: \"9b5d4a9f-0410-49ba-ab07-c0c69e3fe45f\") " pod="kube-system/kas-connection-checker-685895c7c9-kbclw" Oct 30 18:37:26.848662 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.848615 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1340fb2d-5f77-4316-a156-f82e0cb1124d-nginx-conf\") pod \"networking-console-plugin-86c478bb88-dl96p\" (UID: \"1340fb2d-5f77-4316-a156-f82e0cb1124d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" Oct 30 18:37:26.848716 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.848674 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj9qm\" (UniqueName: \"kubernetes.io/projected/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7-kube-api-access-wj9qm\") pod \"service-ca-operator-544d749c8c-b2rv6\" (UID: \"1704ffe1-d69c-4dd2-b7f6-5880bf1551f7\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6" Oct 30 18:37:26.848716 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.848708 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-image-registry-private-configuration\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:26.848814 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.848752 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gqkz\" (UniqueName: \"kubernetes.io/projected/0c876fc3-6980-4a99-aed5-7439ca3d6b69-kube-api-access-5gqkz\") pod \"console-operator-7cd56b8dbc-tp7sc\" (UID: \"0c876fc3-6980-4a99-aed5-7439ca3d6b69\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:26.848814 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.848794 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkd85\" (UniqueName: \"kubernetes.io/projected/542d052a-724f-4d98-9be7-7765a8c1d80a-kube-api-access-fkd85\") pod \"ingress-canary-rbmpn\" (UID: \"542d052a-724f-4d98-9be7-7765a8c1d80a\") " pod="openshift-ingress-canary/ingress-canary-rbmpn" Oct 30 18:37:26.848906 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.848817 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-stats-auth\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:26.848906 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.848842 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00d3be28-5482-48ea-a1be-629e8941236e-tmp\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:26.848906 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.848873 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/13bb697e-8bff-4a30-9e1d-b9f3baac416e-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-gktbq\" (UID: \"13bb697e-8bff-4a30-9e1d-b9f3baac416e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq" Oct 30 18:37:26.848906 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.848875 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/13bb697e-8bff-4a30-9e1d-b9f3baac416e-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-gktbq\" (UID: \"13bb697e-8bff-4a30-9e1d-b9f3baac416e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq" Oct 30 18:37:26.849121 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.848925 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:26.849121 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.848959 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:26.849121 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.848989 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-bound-sa-token\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:26.849121 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849037 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/00d3be28-5482-48ea-a1be-629e8941236e-snapshots\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:26.849121 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849065 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-ca-trust-extracted\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:26.849121 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849088 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-tmp-dir\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:26.849403 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849132 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-certificates\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:26.849403 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849164 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d3be28-5482-48ea-a1be-629e8941236e-service-ca-bundle\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:26.849403 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849189 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-ca-trust-extracted\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:26.849403 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849217 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t24cp\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-kube-api-access-t24cp\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:26.849403 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849250 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q87cx\" (UniqueName: \"kubernetes.io/projected/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-kube-api-access-q87cx\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:26.849403 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849281 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:26.849403 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849311 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6k6g\" (UniqueName: \"kubernetes.io/projected/2297f719-7e19-4de8-b59c-f75885db9cc6-kube-api-access-v6k6g\") pod \"volume-data-source-validator-58c9f5f46d-nprgd\" (UID: \"2297f719-7e19-4de8-b59c-f75885db9cc6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-nprgd" Oct 30 18:37:26.849403 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849335 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c876fc3-6980-4a99-aed5-7439ca3d6b69-trusted-ca\") pod \"console-operator-7cd56b8dbc-tp7sc\" (UID: \"0c876fc3-6980-4a99-aed5-7439ca3d6b69\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:26.849403 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849392 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1340fb2d-5f77-4316-a156-f82e0cb1124d-nginx-conf\") pod \"networking-console-plugin-86c478bb88-dl96p\" (UID: \"1340fb2d-5f77-4316-a156-f82e0cb1124d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" Oct 30 18:37:26.849790 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849422 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5glk7\" (UniqueName: \"kubernetes.io/projected/13bb697e-8bff-4a30-9e1d-b9f3baac416e-kube-api-access-5glk7\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-gktbq\" (UID: \"13bb697e-8bff-4a30-9e1d-b9f3baac416e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq" Oct 30 18:37:26.849790 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849481 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-installation-pull-secrets\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:26.849790 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849624 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:26.849790 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849700 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4q88\" (UniqueName: \"kubernetes.io/projected/00d3be28-5482-48ea-a1be-629e8941236e-kube-api-access-k4q88\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:26.849790 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849732 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-default-certificate\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:26.849790 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849757 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7-config\") pod \"service-ca-operator-544d749c8c-b2rv6\" (UID: \"1704ffe1-d69c-4dd2-b7f6-5880bf1551f7\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6" Oct 30 18:37:26.850076 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849825 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d3be28-5482-48ea-a1be-629e8941236e-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:26.850076 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849856 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7-serving-cert\") pod \"service-ca-operator-544d749c8c-b2rv6\" (UID: \"1704ffe1-d69c-4dd2-b7f6-5880bf1551f7\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6" Oct 30 18:37:26.850076 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849900 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:26.850076 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849955 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/74153c90-cd72-43b8-85c5-aadb4d66a3e4-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:26.850076 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.849989 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flw7z\" (UniqueName: \"kubernetes.io/projected/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-kube-api-access-flw7z\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:26.850076 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.850039 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-installation-pull-secrets\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:26.853051 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.853027 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/13bb697e-8bff-4a30-9e1d-b9f3baac416e-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-gktbq\" (UID: \"13bb697e-8bff-4a30-9e1d-b9f3baac416e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq" Oct 30 18:37:26.856618 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.856589 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Oct 30 18:37:26.856721 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.856705 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Oct 30 18:37:26.856894 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.856878 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Oct 30 18:37:26.856969 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.856955 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Oct 30 18:37:26.857041 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.856978 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Oct 30 18:37:26.857041 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.856988 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Oct 30 18:37:26.857041 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.857032 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-5hnsq\"" Oct 30 18:37:26.857249 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.857072 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Oct 30 18:37:26.857249 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.857115 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Oct 30 18:37:26.857249 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.857127 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Oct 30 18:37:26.857249 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.857159 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Oct 30 18:37:26.857249 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.857167 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-tbwc5\"" Oct 30 18:37:26.857249 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.857192 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Oct 30 18:37:26.857249 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.857201 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Oct 30 18:37:26.857249 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.857075 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 18:37:26.857249 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.857238 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-hvfl9\"" Oct 30 18:37:26.857249 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.857202 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-m4f62\"" Oct 30 18:37:26.857723 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.857301 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-4frj9\"" Oct 30 18:37:26.857723 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.857316 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Oct 30 18:37:26.857723 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.857335 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-db58r\"" Oct 30 18:37:26.857723 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.857380 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 18:37:26.859172 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:26.859143 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:37:26.859267 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.859200 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-fv5kc\"" Oct 30 18:37:26.859840 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:26.859820 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert podName:1340fb2d-5f77-4316-a156-f82e0cb1124d nodeName:}" failed. No retries permitted until 2026-10-30 18:37:27.359679888 +0000 UTC m=+31.670225780 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-dl96p" (UID: "1340fb2d-5f77-4316-a156-f82e0cb1124d") : secret "networking-console-plugin-cert" not found Oct 30 18:37:26.860340 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.860310 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Oct 30 18:37:26.860505 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.860484 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-s6dvj\"" Oct 30 18:37:26.860833 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.860818 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7kh4z\"" Oct 30 18:37:26.861042 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.861026 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-smlbs\"" Oct 30 18:37:26.861093 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.861073 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-q9rgp"] Oct 30 18:37:26.862884 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.862859 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-649485556b-7x5gc"] Oct 30 18:37:26.864081 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.864061 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-d9fjm"] Oct 30 18:37:26.864652 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.864637 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5df4b54dc9-8pdtm"] Oct 30 18:37:26.866563 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.866549 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-nprgd"] Oct 30 18:37:26.867128 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.867114 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6"] Oct 30 18:37:26.893178 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.893159 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rbmpn"] Oct 30 18:37:26.907442 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.907423 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:37:26.907521 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.907494 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:37:26.907830 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.907818 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Oct 30 18:37:26.907914 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.907899 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 18:37:26.907995 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.907982 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Oct 30 18:37:26.908044 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.908033 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Oct 30 18:37:26.908179 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.908168 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:37:26.908219 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.908195 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\"" Oct 30 18:37:26.908219 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.908210 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:37:26.908286 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.908219 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 18:37:26.908317 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.908300 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Oct 30 18:37:26.908596 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.908585 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Oct 30 18:37:26.908817 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.908806 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 18:37:26.908897 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.908887 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Oct 30 18:37:26.909075 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.909063 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 18:37:26.910017 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.909990 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Oct 30 18:37:26.913361 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.913336 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8j7j\" (UniqueName: \"kubernetes.io/projected/6050488d-f6ab-4bab-b2b2-90d30f64bab4-kube-api-access-d8j7j\") pod \"kas-connection-checker-685895c7c9-mzqcg\" (UID: \"6050488d-f6ab-4bab-b2b2-90d30f64bab4\") " pod="kube-system/kas-connection-checker-685895c7c9-mzqcg" Oct 30 18:37:26.913361 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.913357 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twdrz\" (UniqueName: \"kubernetes.io/projected/9b5d4a9f-0410-49ba-ab07-c0c69e3fe45f-kube-api-access-twdrz\") pod \"kas-connection-checker-685895c7c9-kbclw\" (UID: \"9b5d4a9f-0410-49ba-ab07-c0c69e3fe45f\") " pod="kube-system/kas-connection-checker-685895c7c9-kbclw" Oct 30 18:37:26.913467 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.913405 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lnmb\" (UniqueName: \"kubernetes.io/projected/14b832bf-1a7b-4539-811d-84a11d2b55b6-kube-api-access-9lnmb\") pod \"kas-connection-checker-685895c7c9-k4dpj\" (UID: \"14b832bf-1a7b-4539-811d-84a11d2b55b6\") " pod="kube-system/kas-connection-checker-685895c7c9-k4dpj" Oct 30 18:37:26.913467 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.913405 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5glk7\" (UniqueName: \"kubernetes.io/projected/13bb697e-8bff-4a30-9e1d-b9f3baac416e-kube-api-access-5glk7\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-gktbq\" (UID: \"13bb697e-8bff-4a30-9e1d-b9f3baac416e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq" Oct 30 18:37:26.944250 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.944233 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-k4dpj" Oct 30 18:37:26.949182 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.949163 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-mzqcg" Oct 30 18:37:26.950377 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950362 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-metrics-certs\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:26.950438 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950388 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/00d3be28-5482-48ea-a1be-629e8941236e-serving-cert\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:26.950438 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950406 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-image-registry-private-configuration\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:26.950438 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950423 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-trusted-ca\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:26.950563 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950438 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w86l\" (UniqueName: \"kubernetes.io/projected/74153c90-cd72-43b8-85c5-aadb4d66a3e4-kube-api-access-8w86l\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:26.950563 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950453 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:26.950563 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950469 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw7kf\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-kube-api-access-dw7kf\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:26.950563 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950487 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0c876fc3-6980-4a99-aed5-7439ca3d6b69-config\") pod \"console-operator-7cd56b8dbc-tp7sc\" (UID: \"0c876fc3-6980-4a99-aed5-7439ca3d6b69\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:26.950563 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950508 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-trusted-ca\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:26.950563 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:26.950527 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 18:37:26.950563 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950549 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0c876fc3-6980-4a99-aed5-7439ca3d6b69-serving-cert\") pod \"console-operator-7cd56b8dbc-tp7sc\" (UID: \"0c876fc3-6980-4a99-aed5-7439ca3d6b69\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:26.950883 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950570 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/542d052a-724f-4d98-9be7-7765a8c1d80a-cert\") pod \"ingress-canary-rbmpn\" (UID: \"542d052a-724f-4d98-9be7-7765a8c1d80a\") " pod="openshift-ingress-canary/ingress-canary-rbmpn" Oct 30 18:37:26.950883 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:26.950587 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:37:26.950883 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:26.950599 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-metrics-certs podName:8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:27.450582895 +0000 UTC m=+31.761128789 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-metrics-certs") pod "router-default-6c975679c5-x5lg8" (UID: "8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7") : secret "router-metrics-certs-default" not found Oct 30 18:37:26.950883 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950624 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-certificates\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:26.950883 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950660 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zf4n\" (UniqueName: \"kubernetes.io/projected/b18682b8-f91a-4c27-87eb-82720ba4257f-kube-api-access-5zf4n\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:26.950883 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950685 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-bound-sa-token\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:26.950883 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950711 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjf45\" (UniqueName: \"kubernetes.io/projected/68788658-4afb-4757-acfa-7638cfb2488b-kube-api-access-mjf45\") pod \"network-check-source-6f78d59f99-d9fjm\" (UID: \"68788658-4afb-4757-acfa-7638cfb2488b\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-d9fjm" Oct 30 18:37:26.950883 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950750 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:26.950883 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950789 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj9qm\" (UniqueName: \"kubernetes.io/projected/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7-kube-api-access-wj9qm\") pod \"service-ca-operator-544d749c8c-b2rv6\" (UID: \"1704ffe1-d69c-4dd2-b7f6-5880bf1551f7\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6" Oct 30 18:37:26.950883 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950813 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-image-registry-private-configuration\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:26.950883 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950843 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gqkz\" (UniqueName: \"kubernetes.io/projected/0c876fc3-6980-4a99-aed5-7439ca3d6b69-kube-api-access-5gqkz\") pod \"console-operator-7cd56b8dbc-tp7sc\" (UID: \"0c876fc3-6980-4a99-aed5-7439ca3d6b69\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:26.950883 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950871 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkd85\" (UniqueName: \"kubernetes.io/projected/542d052a-724f-4d98-9be7-7765a8c1d80a-kube-api-access-fkd85\") pod \"ingress-canary-rbmpn\" (UID: \"542d052a-724f-4d98-9be7-7765a8c1d80a\") " pod="openshift-ingress-canary/ingress-canary-rbmpn" Oct 30 18:37:26.951477 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950898 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-stats-auth\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:26.951477 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950920 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00d3be28-5482-48ea-a1be-629e8941236e-tmp\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:26.951477 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950950 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:26.951477 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.950977 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:26.951477 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951024 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-bound-sa-token\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:26.951477 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951050 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/00d3be28-5482-48ea-a1be-629e8941236e-snapshots\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:26.951477 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951083 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-ca-trust-extracted\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:26.951477 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951108 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-tmp-dir\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:26.951477 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951150 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-certificates\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:26.951477 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951181 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d3be28-5482-48ea-a1be-629e8941236e-service-ca-bundle\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:26.951477 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951212 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-ca-trust-extracted\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:26.951477 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951240 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t24cp\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-kube-api-access-t24cp\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:26.951477 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951274 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q87cx\" (UniqueName: \"kubernetes.io/projected/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-kube-api-access-q87cx\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:26.951477 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951303 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:26.951477 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951319 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0c876fc3-6980-4a99-aed5-7439ca3d6b69-config\") pod \"console-operator-7cd56b8dbc-tp7sc\" (UID: \"0c876fc3-6980-4a99-aed5-7439ca3d6b69\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:26.951477 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951332 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6k6g\" (UniqueName: \"kubernetes.io/projected/2297f719-7e19-4de8-b59c-f75885db9cc6-kube-api-access-v6k6g\") pod \"volume-data-source-validator-58c9f5f46d-nprgd\" (UID: \"2297f719-7e19-4de8-b59c-f75885db9cc6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-nprgd" Oct 30 18:37:26.952137 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951368 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c876fc3-6980-4a99-aed5-7439ca3d6b69-trusted-ca\") pod \"console-operator-7cd56b8dbc-tp7sc\" (UID: \"0c876fc3-6980-4a99-aed5-7439ca3d6b69\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:26.952137 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951400 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-installation-pull-secrets\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:26.952137 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951436 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:26.952137 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951469 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4q88\" (UniqueName: \"kubernetes.io/projected/00d3be28-5482-48ea-a1be-629e8941236e-kube-api-access-k4q88\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:26.952137 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951496 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-default-certificate\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:26.952137 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951524 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7-config\") pod \"service-ca-operator-544d749c8c-b2rv6\" (UID: \"1704ffe1-d69c-4dd2-b7f6-5880bf1551f7\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6" Oct 30 18:37:26.952137 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951558 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d3be28-5482-48ea-a1be-629e8941236e-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:26.952137 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951583 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7-serving-cert\") pod \"service-ca-operator-544d749c8c-b2rv6\" (UID: \"1704ffe1-d69c-4dd2-b7f6-5880bf1551f7\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6" Oct 30 18:37:26.952137 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951611 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:26.952137 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951654 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/74153c90-cd72-43b8-85c5-aadb4d66a3e4-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:26.952137 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951682 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flw7z\" (UniqueName: \"kubernetes.io/projected/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-kube-api-access-flw7z\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:26.952137 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951685 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-trusted-ca\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:26.952137 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951708 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-installation-pull-secrets\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:26.952137 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.951735 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-config-volume\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:26.952760 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.952289 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-config-volume\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:26.956615 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.953021 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-image-registry-private-configuration\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:26.956615 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.953128 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-trusted-ca\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:26.956615 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.953539 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-certificates\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:26.956615 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.953647 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7-config\") pod \"service-ca-operator-544d749c8c-b2rv6\" (UID: \"1704ffe1-d69c-4dd2-b7f6-5880bf1551f7\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6" Oct 30 18:37:26.956615 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:26.950602 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5df4b54dc9-8pdtm: secret "image-registry-tls" not found Oct 30 18:37:26.956615 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.953785 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-ca-trust-extracted\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:26.956615 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.953832 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/74153c90-cd72-43b8-85c5-aadb4d66a3e4-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:26.956615 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:26.953823 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls podName:7fcdea8f-3b0f-42c5-bb3d-b2266b046491 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:27.453805458 +0000 UTC m=+31.764351345 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls") pod "image-registry-5df4b54dc9-8pdtm" (UID: "7fcdea8f-3b0f-42c5-bb3d-b2266b046491") : secret "image-registry-tls" not found Oct 30 18:37:26.956615 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.954058 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-tmp-dir\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:26.956615 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:26.951727 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:37:26.956615 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.954374 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/00d3be28-5482-48ea-a1be-629e8941236e-serving-cert\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:26.956615 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:26.954406 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls podName:b18682b8-f91a-4c27-87eb-82720ba4257f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:27.45439154 +0000 UTC m=+31.764937427 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6mt5b" (UID: "b18682b8-f91a-4c27-87eb-82720ba4257f") : secret "samples-operator-tls" not found Oct 30 18:37:26.956615 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:26.953115 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:37:26.956615 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:26.954449 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls podName:447f2b64-f13e-48cc-94b9-e80df4a9d4f3 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:27.454440192 +0000 UTC m=+31.764986067 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls") pod "dns-default-ltkzx" (UID: "447f2b64-f13e-48cc-94b9-e80df4a9d4f3") : secret "dns-default-metrics-tls" not found Oct 30 18:37:26.956615 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.954488 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/00d3be28-5482-48ea-a1be-629e8941236e-snapshots\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:26.956615 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.954511 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-certificates\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:26.958083 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.954823 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-ca-trust-extracted\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:26.958083 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.954876 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/542d052a-724f-4d98-9be7-7765a8c1d80a-cert\") pod \"ingress-canary-rbmpn\" (UID: \"542d052a-724f-4d98-9be7-7765a8c1d80a\") " pod="openshift-ingress-canary/ingress-canary-rbmpn" Oct 30 18:37:26.958083 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:26.954900 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:37:26.958083 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:26.954955 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls podName:74153c90-cd72-43b8-85c5-aadb4d66a3e4 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:27.454940148 +0000 UTC m=+31.765486034 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-cf8qt" (UID: "74153c90-cd72-43b8-85c5-aadb4d66a3e4") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:37:26.958083 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.955367 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00d3be28-5482-48ea-a1be-629e8941236e-tmp\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:26.958083 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.955627 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:26.958083 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.955762 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c876fc3-6980-4a99-aed5-7439ca3d6b69-trusted-ca\") pod \"console-operator-7cd56b8dbc-tp7sc\" (UID: \"0c876fc3-6980-4a99-aed5-7439ca3d6b69\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:26.958083 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:26.955865 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle podName:8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:27.455851983 +0000 UTC m=+31.766397893 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle") pod "router-default-6c975679c5-x5lg8" (UID: "8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7") : configmap references non-existent config key: service-ca.crt Oct 30 18:37:26.958083 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:26.955936 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:37:26.958083 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:26.955951 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649485556b-7x5gc: secret "image-registry-tls" not found Oct 30 18:37:26.958083 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:26.955987 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls podName:d9b465b9-97b6-47e2-8732-ed29b8f24ed0 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:27.455974479 +0000 UTC m=+31.766520357 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls") pod "image-registry-649485556b-7x5gc" (UID: "d9b465b9-97b6-47e2-8732-ed29b8f24ed0") : secret "image-registry-tls" not found Oct 30 18:37:26.958083 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.956379 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-installation-pull-secrets\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:26.958083 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.956560 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d3be28-5482-48ea-a1be-629e8941236e-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:26.958083 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.956742 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d3be28-5482-48ea-a1be-629e8941236e-service-ca-bundle\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:26.958083 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.957413 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-installation-pull-secrets\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:26.958865 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.958159 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7-serving-cert\") pod \"service-ca-operator-544d749c8c-b2rv6\" (UID: \"1704ffe1-d69c-4dd2-b7f6-5880bf1551f7\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6" Oct 30 18:37:26.958865 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.958656 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-image-registry-private-configuration\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:26.958865 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.958841 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0c876fc3-6980-4a99-aed5-7439ca3d6b69-serving-cert\") pod \"console-operator-7cd56b8dbc-tp7sc\" (UID: \"0c876fc3-6980-4a99-aed5-7439ca3d6b69\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:26.959877 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.959783 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-default-certificate\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:26.960283 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.960242 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-stats-auth\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:26.980338 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.980081 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq" Oct 30 18:37:26.985704 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:26.985611 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-kbclw" Oct 30 18:37:27.013686 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.013608 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zf4n\" (UniqueName: \"kubernetes.io/projected/b18682b8-f91a-4c27-87eb-82720ba4257f-kube-api-access-5zf4n\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:27.018613 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.014988 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w86l\" (UniqueName: \"kubernetes.io/projected/74153c90-cd72-43b8-85c5-aadb4d66a3e4-kube-api-access-8w86l\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:27.020515 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.020219 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj9qm\" (UniqueName: \"kubernetes.io/projected/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7-kube-api-access-wj9qm\") pod \"service-ca-operator-544d749c8c-b2rv6\" (UID: \"1704ffe1-d69c-4dd2-b7f6-5880bf1551f7\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6" Oct 30 18:37:27.030096 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.028625 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjf45\" (UniqueName: \"kubernetes.io/projected/68788658-4afb-4757-acfa-7638cfb2488b-kube-api-access-mjf45\") pod \"network-check-source-6f78d59f99-d9fjm\" (UID: \"68788658-4afb-4757-acfa-7638cfb2488b\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-d9fjm" Oct 30 18:37:27.030096 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.029198 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw7kf\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-kube-api-access-dw7kf\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:27.032536 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.032507 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-bound-sa-token\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:27.035642 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.035611 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkd85\" (UniqueName: \"kubernetes.io/projected/542d052a-724f-4d98-9be7-7765a8c1d80a-kube-api-access-fkd85\") pod \"ingress-canary-rbmpn\" (UID: \"542d052a-724f-4d98-9be7-7765a8c1d80a\") " pod="openshift-ingress-canary/ingress-canary-rbmpn" Oct 30 18:37:27.037427 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.037276 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gqkz\" (UniqueName: \"kubernetes.io/projected/0c876fc3-6980-4a99-aed5-7439ca3d6b69-kube-api-access-5gqkz\") pod \"console-operator-7cd56b8dbc-tp7sc\" (UID: \"0c876fc3-6980-4a99-aed5-7439ca3d6b69\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:27.038252 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.038204 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q87cx\" (UniqueName: \"kubernetes.io/projected/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-kube-api-access-q87cx\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:27.038898 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.038860 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flw7z\" (UniqueName: \"kubernetes.io/projected/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-kube-api-access-flw7z\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:27.043537 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.041215 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6k6g\" (UniqueName: \"kubernetes.io/projected/2297f719-7e19-4de8-b59c-f75885db9cc6-kube-api-access-v6k6g\") pod \"volume-data-source-validator-58c9f5f46d-nprgd\" (UID: \"2297f719-7e19-4de8-b59c-f75885db9cc6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-nprgd" Oct 30 18:37:27.043537 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.042818 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t24cp\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-kube-api-access-t24cp\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:27.045050 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.045023 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4q88\" (UniqueName: \"kubernetes.io/projected/00d3be28-5482-48ea-a1be-629e8941236e-kube-api-access-k4q88\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:27.049938 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.049914 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-bound-sa-token\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:27.061731 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.061518 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-nprgd" Oct 30 18:37:27.066913 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.066382 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6" Oct 30 18:37:27.074188 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.073165 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:27.087025 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.086989 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-d9fjm" Oct 30 18:37:27.093127 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.092968 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rbmpn" Oct 30 18:37:27.182702 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.181790 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:27.233780 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.233704 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-smcdv\"" Oct 30 18:37:27.234129 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.233849 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 18:37:27.305879 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.305854 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:27.354728 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.354697 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-mzqcg"] Oct 30 18:37:27.361463 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.361439 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-dl96p\" (UID: \"1340fb2d-5f77-4316-a156-f82e0cb1124d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" Oct 30 18:37:27.361618 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:27.361598 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:37:27.361683 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:27.361673 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert podName:1340fb2d-5f77-4316-a156-f82e0cb1124d nodeName:}" failed. No retries permitted until 2026-10-30 18:37:28.361654967 +0000 UTC m=+32.672200846 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-dl96p" (UID: "1340fb2d-5f77-4316-a156-f82e0cb1124d") : secret "networking-console-plugin-cert" not found Oct 30 18:37:27.362231 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.362216 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-k4dpj"] Oct 30 18:37:27.369192 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:37:27.369161 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6050488d_f6ab_4bab_b2b2_90d30f64bab4.slice/crio-e43a7a8c7a9b211e176baf848feebaf574c585874ab96cffee7861f821b25402 WatchSource:0}: Error finding container e43a7a8c7a9b211e176baf848feebaf574c585874ab96cffee7861f821b25402: Status 404 returned error can't find the container with id e43a7a8c7a9b211e176baf848feebaf574c585874ab96cffee7861f821b25402 Oct 30 18:37:27.382556 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.382530 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-kbclw"] Oct 30 18:37:27.384607 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.384585 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq"] Oct 30 18:37:27.399737 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:37:27.399708 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13bb697e_8bff_4a30_9e1d_b9f3baac416e.slice/crio-06aaf11fadf294f562fe5dfead8760ff636dcd908ca84d12a367ccabbb14f879 WatchSource:0}: Error finding container 06aaf11fadf294f562fe5dfead8760ff636dcd908ca84d12a367ccabbb14f879: Status 404 returned error can't find the container with id 06aaf11fadf294f562fe5dfead8760ff636dcd908ca84d12a367ccabbb14f879 Oct 30 18:37:27.400919 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:37:27.400891 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b5d4a9f_0410_49ba_ab07_c0c69e3fe45f.slice/crio-784749f8c586b5ddfc3e84a7905c2212d47aa63e39a66a8277afbcd91d228b18 WatchSource:0}: Error finding container 784749f8c586b5ddfc3e84a7905c2212d47aa63e39a66a8277afbcd91d228b18: Status 404 returned error can't find the container with id 784749f8c586b5ddfc3e84a7905c2212d47aa63e39a66a8277afbcd91d228b18 Oct 30 18:37:27.462346 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.462321 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:27.462483 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.462368 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-metrics-certs\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:27.462483 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.462408 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:27.462483 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.462469 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:27.462631 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.462527 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:27.462631 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.462554 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:27.462631 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.462624 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:27.462801 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:27.462725 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:37:27.462801 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:27.462776 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls podName:74153c90-cd72-43b8-85c5-aadb4d66a3e4 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:28.462758958 +0000 UTC m=+32.773304836 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-cf8qt" (UID: "74153c90-cd72-43b8-85c5-aadb4d66a3e4") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:37:27.463038 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:27.462830 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:37:27.463038 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:27.462856 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle podName:8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:28.462844067 +0000 UTC m=+32.773389958 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle") pod "router-default-6c975679c5-x5lg8" (UID: "8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7") : configmap references non-existent config key: service-ca.crt Oct 30 18:37:27.463038 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:27.462867 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:37:27.463038 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:27.462842 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 18:37:27.463038 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:27.462889 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:37:27.463038 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:27.462909 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649485556b-7x5gc: secret "image-registry-tls" not found Oct 30 18:37:27.463038 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:27.462892 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls podName:447f2b64-f13e-48cc-94b9-e80df4a9d4f3 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:28.462876412 +0000 UTC m=+32.773422313 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls") pod "dns-default-ltkzx" (UID: "447f2b64-f13e-48cc-94b9-e80df4a9d4f3") : secret "dns-default-metrics-tls" not found Oct 30 18:37:27.463038 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:27.462960 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:37:27.463038 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:27.462969 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5df4b54dc9-8pdtm: secret "image-registry-tls" not found Oct 30 18:37:27.463038 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:27.462961 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls podName:b18682b8-f91a-4c27-87eb-82720ba4257f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:28.462931666 +0000 UTC m=+32.773477558 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6mt5b" (UID: "b18682b8-f91a-4c27-87eb-82720ba4257f") : secret "samples-operator-tls" not found Oct 30 18:37:27.463038 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:27.463033 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-metrics-certs podName:8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:28.463021433 +0000 UTC m=+32.773567314 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-metrics-certs") pod "router-default-6c975679c5-x5lg8" (UID: "8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7") : secret "router-metrics-certs-default" not found Oct 30 18:37:27.463363 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:27.463050 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls podName:d9b465b9-97b6-47e2-8732-ed29b8f24ed0 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:28.463045401 +0000 UTC m=+32.773591276 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls") pod "image-registry-649485556b-7x5gc" (UID: "d9b465b9-97b6-47e2-8732-ed29b8f24ed0") : secret "image-registry-tls" not found Oct 30 18:37:27.463363 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:27.463063 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls podName:7fcdea8f-3b0f-42c5-bb3d-b2266b046491 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:28.463058642 +0000 UTC m=+32.773604516 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls") pod "image-registry-5df4b54dc9-8pdtm" (UID: "7fcdea8f-3b0f-42c5-bb3d-b2266b046491") : secret "image-registry-tls" not found Oct 30 18:37:27.468165 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.468097 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-nprgd"] Oct 30 18:37:27.474578 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:37:27.474547 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2297f719_7e19_4de8_b59c_f75885db9cc6.slice/crio-c94e473384b14578f6885f47d13268485b29d20ed4b69f3881eb006a56e5984c WatchSource:0}: Error finding container c94e473384b14578f6885f47d13268485b29d20ed4b69f3881eb006a56e5984c: Status 404 returned error can't find the container with id c94e473384b14578f6885f47d13268485b29d20ed4b69f3881eb006a56e5984c Oct 30 18:37:27.482533 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.482462 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6"] Oct 30 18:37:27.496973 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.495818 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-tp7sc"] Oct 30 18:37:27.507780 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.505761 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-d9fjm"] Oct 30 18:37:27.507780 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.507149 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rbmpn"] Oct 30 18:37:27.674119 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:27.674093 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-q9rgp"] Oct 30 18:37:27.680778 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:37:27.680750 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00d3be28_5482_48ea_a1be_629e8941236e.slice/crio-6c336c6ab08a5c01702f4bee9b09fa45602c34a124233ec54a5394f16bcbd95d WatchSource:0}: Error finding container 6c336c6ab08a5c01702f4bee9b09fa45602c34a124233ec54a5394f16bcbd95d: Status 404 returned error can't find the container with id 6c336c6ab08a5c01702f4bee9b09fa45602c34a124233ec54a5394f16bcbd95d Oct 30 18:37:28.181749 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.181717 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:28.181938 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.181892 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:28.233280 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.233261 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 18:37:28.233460 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.233424 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-gsdbf\"" Oct 30 18:37:28.233544 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.233532 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w4fd8\"" Oct 30 18:37:28.371634 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.371600 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-dl96p\" (UID: \"1340fb2d-5f77-4316-a156-f82e0cb1124d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" Oct 30 18:37:28.372066 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:28.371742 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:37:28.372066 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:28.371794 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert podName:1340fb2d-5f77-4316-a156-f82e0cb1124d nodeName:}" failed. No retries permitted until 2026-10-30 18:37:30.371776941 +0000 UTC m=+34.682322817 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-dl96p" (UID: "1340fb2d-5f77-4316-a156-f82e0cb1124d") : secret "networking-console-plugin-cert" not found Oct 30 18:37:28.372987 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.372590 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-kbclw" event={"ID":"9b5d4a9f-0410-49ba-ab07-c0c69e3fe45f","Type":"ContainerStarted","Data":"081a3f4290698b9c22c2e4e07c869ef90c017a6970f48643bcb9b39400a7d517"} Oct 30 18:37:28.372987 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.372643 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-kbclw" event={"ID":"9b5d4a9f-0410-49ba-ab07-c0c69e3fe45f","Type":"ContainerStarted","Data":"784749f8c586b5ddfc3e84a7905c2212d47aa63e39a66a8277afbcd91d228b18"} Oct 30 18:37:28.374118 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.374091 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-k4dpj" event={"ID":"14b832bf-1a7b-4539-811d-84a11d2b55b6","Type":"ContainerStarted","Data":"da69f2e5099e475a831f9f8aba4edd0fda786298907c59ff1aadfdc53836edfc"} Oct 30 18:37:28.374239 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.374130 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-k4dpj" event={"ID":"14b832bf-1a7b-4539-811d-84a11d2b55b6","Type":"ContainerStarted","Data":"4dcdea706b660db0b90895641b499973489fd436605a40714e2efc108462bb7a"} Oct 30 18:37:28.375728 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.375694 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6" event={"ID":"1704ffe1-d69c-4dd2-b7f6-5880bf1551f7","Type":"ContainerStarted","Data":"c286ec7589887ecf4b20167c2733b290cc9be04fb5b0479abab84f075c47828e"} Oct 30 18:37:28.376990 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.376967 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq" event={"ID":"13bb697e-8bff-4a30-9e1d-b9f3baac416e","Type":"ContainerStarted","Data":"06aaf11fadf294f562fe5dfead8760ff636dcd908ca84d12a367ccabbb14f879"} Oct 30 18:37:28.378604 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.378578 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-mzqcg" event={"ID":"6050488d-f6ab-4bab-b2b2-90d30f64bab4","Type":"ContainerStarted","Data":"31ffff3f34fc13cb50457df65ca867b0b96bc8cdfb62b73591c1baabe139148f"} Oct 30 18:37:28.378749 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.378616 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-mzqcg" event={"ID":"6050488d-f6ab-4bab-b2b2-90d30f64bab4","Type":"ContainerStarted","Data":"e43a7a8c7a9b211e176baf848feebaf574c585874ab96cffee7861f821b25402"} Oct 30 18:37:28.380034 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.379992 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-q9rgp" event={"ID":"00d3be28-5482-48ea-a1be-629e8941236e","Type":"ContainerStarted","Data":"6c336c6ab08a5c01702f4bee9b09fa45602c34a124233ec54a5394f16bcbd95d"} Oct 30 18:37:28.381317 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.381295 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-nprgd" event={"ID":"2297f719-7e19-4de8-b59c-f75885db9cc6","Type":"ContainerStarted","Data":"c94e473384b14578f6885f47d13268485b29d20ed4b69f3881eb006a56e5984c"} Oct 30 18:37:28.382577 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.382552 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" event={"ID":"0c876fc3-6980-4a99-aed5-7439ca3d6b69","Type":"ContainerStarted","Data":"7f59f42912f97c4914944345a46a250dbad8bda167a07f70972f2addbf6dc9b1"} Oct 30 18:37:28.383797 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.383754 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rbmpn" event={"ID":"542d052a-724f-4d98-9be7-7765a8c1d80a","Type":"ContainerStarted","Data":"2f375283f7767fa35504e3aca70a46b4db0284adef37d35bff11ce72f10ac3d2"} Oct 30 18:37:28.384966 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.384944 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-d9fjm" event={"ID":"68788658-4afb-4757-acfa-7638cfb2488b","Type":"ContainerStarted","Data":"cc7cd7288990cb2519f5778f0589c8defb838504559335fca0d063c1d4160fb9"} Oct 30 18:37:28.472151 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.472081 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:28.472151 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.472149 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-metrics-certs\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:28.472384 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.472190 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:28.472384 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.472271 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:28.472384 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.472348 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:28.472540 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.472378 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:28.472540 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.472474 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:28.472630 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:28.472601 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:37:28.472679 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:28.472667 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls podName:74153c90-cd72-43b8-85c5-aadb4d66a3e4 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:30.472649084 +0000 UTC m=+34.783194959 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-cf8qt" (UID: "74153c90-cd72-43b8-85c5-aadb4d66a3e4") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:37:28.472756 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:28.472742 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:37:28.472828 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:28.472779 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls podName:447f2b64-f13e-48cc-94b9-e80df4a9d4f3 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:30.472769578 +0000 UTC m=+34.783315461 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls") pod "dns-default-ltkzx" (UID: "447f2b64-f13e-48cc-94b9-e80df4a9d4f3") : secret "dns-default-metrics-tls" not found Oct 30 18:37:28.473292 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:28.473269 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:37:28.473401 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:28.473320 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls podName:b18682b8-f91a-4c27-87eb-82720ba4257f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:30.473308339 +0000 UTC m=+34.783854230 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6mt5b" (UID: "b18682b8-f91a-4c27-87eb-82720ba4257f") : secret "samples-operator-tls" not found Oct 30 18:37:28.473401 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:28.473378 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 18:37:28.473482 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:28.473411 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-metrics-certs podName:8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:30.473399904 +0000 UTC m=+34.783945794 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-metrics-certs") pod "router-default-6c975679c5-x5lg8" (UID: "8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7") : secret "router-metrics-certs-default" not found Oct 30 18:37:28.473482 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:28.473475 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:37:28.473546 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:28.473486 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5df4b54dc9-8pdtm: secret "image-registry-tls" not found Oct 30 18:37:28.473546 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:28.473517 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls podName:7fcdea8f-3b0f-42c5-bb3d-b2266b046491 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:30.473507037 +0000 UTC m=+34.784052913 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls") pod "image-registry-5df4b54dc9-8pdtm" (UID: "7fcdea8f-3b0f-42c5-bb3d-b2266b046491") : secret "image-registry-tls" not found Oct 30 18:37:28.473974 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:28.473954 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:37:28.473974 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:28.473973 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649485556b-7x5gc: secret "image-registry-tls" not found Oct 30 18:37:28.474130 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:28.474023 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls podName:d9b465b9-97b6-47e2-8732-ed29b8f24ed0 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:30.473996854 +0000 UTC m=+34.784542734 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls") pod "image-registry-649485556b-7x5gc" (UID: "d9b465b9-97b6-47e2-8732-ed29b8f24ed0") : secret "image-registry-tls" not found Oct 30 18:37:28.474430 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:28.474412 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle podName:8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:30.474402176 +0000 UTC m=+34.784948068 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle") pod "router-default-6c975679c5-x5lg8" (UID: "8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7") : configmap references non-existent config key: service-ca.crt Oct 30 18:37:28.879516 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:28.879461 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-k4dpj" podStartSLOduration=348.879440722 podStartE2EDuration="5m48.879440722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:28.767370139 +0000 UTC m=+33.077916037" watchObservedRunningTime="2026-10-30 18:37:28.879440722 +0000 UTC m=+33.189986621" Oct 30 18:37:29.008687 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:29.008627 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-kbclw" podStartSLOduration=349.008607787 podStartE2EDuration="5m49.008607787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:28.881021412 +0000 UTC m=+33.191567299" watchObservedRunningTime="2026-10-30 18:37:29.008607787 +0000 UTC m=+33.319153685" Oct 30 18:37:29.988132 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:29.988049 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs\") pod \"network-metrics-daemon-phngs\" (UID: \"25bce85b-a92c-4c39-a298-7b1881380069\") " pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:29.988132 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:29.988120 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:29.988626 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:29.988543 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Oct 30 18:37:29.988626 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:29.988624 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs podName:25bce85b-a92c-4c39-a298-7b1881380069 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:01.988597419 +0000 UTC m=+66.299143313 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs") pod "network-metrics-daemon-phngs" (UID: "25bce85b-a92c-4c39-a298-7b1881380069") : secret "metrics-daemon-secret" not found Oct 30 18:37:30.011685 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:30.011658 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:30.089969 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:30.089515 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr22p\" (UniqueName: \"kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p\") pod \"network-check-target-62vfn\" (UID: \"97294b18-463b-429d-8073-dc532ab0a046\") " pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:30.096423 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:30.096356 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr22p\" (UniqueName: \"kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p\") pod \"network-check-target-62vfn\" (UID: \"97294b18-463b-429d-8073-dc532ab0a046\") " pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:30.197595 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:30.197564 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:30.299197 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:30.299161 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:30.402942 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:30.402366 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-dl96p\" (UID: \"1340fb2d-5f77-4316-a156-f82e0cb1124d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" Oct 30 18:37:30.402942 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:30.402519 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:37:30.402942 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:30.402590 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert podName:1340fb2d-5f77-4316-a156-f82e0cb1124d nodeName:}" failed. No retries permitted until 2026-10-30 18:37:34.402571187 +0000 UTC m=+38.713117063 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-dl96p" (UID: "1340fb2d-5f77-4316-a156-f82e0cb1124d") : secret "networking-console-plugin-cert" not found Oct 30 18:37:30.409806 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:30.409778 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-nprgd" event={"ID":"2297f719-7e19-4de8-b59c-f75885db9cc6","Type":"ContainerStarted","Data":"52cc3f6d270677ae543c7d06097398081ada72567cd1b215ec3e1122fb397bc9"} Oct 30 18:37:30.503613 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:30.503539 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:30.503758 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:30.503661 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-metrics-certs\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:30.503758 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:30.503701 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:30.503865 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:30.503775 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 18:37:30.503865 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:30.503842 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:37:30.503955 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:30.503846 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-metrics-certs podName:8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:34.503826522 +0000 UTC m=+38.814372410 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-metrics-certs") pod "router-default-6c975679c5-x5lg8" (UID: "8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7") : secret "router-metrics-certs-default" not found Oct 30 18:37:30.503955 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:30.503890 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls podName:447f2b64-f13e-48cc-94b9-e80df4a9d4f3 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:34.503878102 +0000 UTC m=+38.814423982 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls") pod "dns-default-ltkzx" (UID: "447f2b64-f13e-48cc-94b9-e80df4a9d4f3") : secret "dns-default-metrics-tls" not found Oct 30 18:37:30.503955 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:30.503953 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:37:30.504115 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:30.503964 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5df4b54dc9-8pdtm: secret "image-registry-tls" not found Oct 30 18:37:30.504115 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:30.503995 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls podName:7fcdea8f-3b0f-42c5-bb3d-b2266b046491 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:34.503985047 +0000 UTC m=+38.814530936 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls") pod "image-registry-5df4b54dc9-8pdtm" (UID: "7fcdea8f-3b0f-42c5-bb3d-b2266b046491") : secret "image-registry-tls" not found Oct 30 18:37:30.504228 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:30.504133 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:30.504228 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:30.504202 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:30.504329 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:30.504234 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:30.504381 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:30.504317 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:30.504381 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:30.504327 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:37:30.504476 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:30.504402 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle podName:8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:34.504388674 +0000 UTC m=+38.814934569 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle") pod "router-default-6c975679c5-x5lg8" (UID: "8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7") : configmap references non-existent config key: service-ca.crt Oct 30 18:37:30.504476 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:30.504423 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls podName:b18682b8-f91a-4c27-87eb-82720ba4257f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:34.504414109 +0000 UTC m=+38.814959990 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6mt5b" (UID: "b18682b8-f91a-4c27-87eb-82720ba4257f") : secret "samples-operator-tls" not found Oct 30 18:37:30.504585 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:30.504478 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:37:30.504585 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:30.504489 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649485556b-7x5gc: secret "image-registry-tls" not found Oct 30 18:37:30.504585 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:30.504518 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls podName:d9b465b9-97b6-47e2-8732-ed29b8f24ed0 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:34.504509128 +0000 UTC m=+38.815055036 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls") pod "image-registry-649485556b-7x5gc" (UID: "d9b465b9-97b6-47e2-8732-ed29b8f24ed0") : secret "image-registry-tls" not found Oct 30 18:37:30.504729 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:30.504590 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:37:30.504729 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:30.504620 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls podName:74153c90-cd72-43b8-85c5-aadb4d66a3e4 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:34.504610607 +0000 UTC m=+38.815156486 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-cf8qt" (UID: "74153c90-cd72-43b8-85c5-aadb4d66a3e4") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:37:30.522724 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:30.522675 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-nprgd" podStartSLOduration=319.382158723 podStartE2EDuration="5m21.522660926s" totalImagesPullingTime="2.140502203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:32:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:30.522208459 +0000 UTC m=+34.832754357" watchObservedRunningTime="2026-10-30 18:37:30.522660926 +0000 UTC m=+34.833206832" Oct 30 18:37:30.523476 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:30.523429 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-mzqcg" podStartSLOduration=350.523420449 podStartE2EDuration="5m50.523420449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:31:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:29.010155509 +0000 UTC m=+33.320701407" watchObservedRunningTime="2026-10-30 18:37:30.523420449 +0000 UTC m=+34.833966347" Oct 30 18:37:30.636294 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:30.636261 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pxndl"] Oct 30 18:37:30.752514 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:30.752483 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-62vfn"] Oct 30 18:37:30.761293 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:37:30.761264 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97294b18_463b_429d_8073_dc532ab0a046.slice/crio-d436d35f5c3b1730e19584bb15e329a39f0bb973334d2e7385433323968199f9 WatchSource:0}: Error finding container d436d35f5c3b1730e19584bb15e329a39f0bb973334d2e7385433323968199f9: Status 404 returned error can't find the container with id d436d35f5c3b1730e19584bb15e329a39f0bb973334d2e7385433323968199f9 Oct 30 18:37:31.415651 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:31.415572 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-62vfn" event={"ID":"97294b18-463b-429d-8073-dc532ab0a046","Type":"ContainerStarted","Data":"d436d35f5c3b1730e19584bb15e329a39f0bb973334d2e7385433323968199f9"} Oct 30 18:37:31.418654 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:31.418622 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pxndl" event={"ID":"f1cae505-4cef-46e5-85cc-cf8b410e654a","Type":"ContainerStarted","Data":"3c8a0a6850f3ad80a626af13fa06f6b320b58224072762baa98b8e510c93131f"} Oct 30 18:37:33.429549 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:33.428788 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-q9rgp" event={"ID":"00d3be28-5482-48ea-a1be-629e8941236e","Type":"ContainerStarted","Data":"52e79f8bba3ae526daf24de718556caf07d88454dbd395f74b1ccfe79a7b4d81"} Oct 30 18:37:33.431718 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:33.431687 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/0.log" Oct 30 18:37:33.431842 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:33.431728 2581 generic.go:356] "Generic (PLEG): container finished" podID="0c876fc3-6980-4a99-aed5-7439ca3d6b69" containerID="e24b247eaedd205eb67d2516feac687e8dc32c329c6fcad6ac4dd7d07dc8a821" exitCode=255 Oct 30 18:37:33.431842 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:33.431814 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" event={"ID":"0c876fc3-6980-4a99-aed5-7439ca3d6b69","Type":"ContainerDied","Data":"e24b247eaedd205eb67d2516feac687e8dc32c329c6fcad6ac4dd7d07dc8a821"} Oct 30 18:37:33.432121 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:33.432106 2581 scope.go:122] "RemoveContainer" containerID="e24b247eaedd205eb67d2516feac687e8dc32c329c6fcad6ac4dd7d07dc8a821" Oct 30 18:37:33.436454 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:33.436430 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6" event={"ID":"1704ffe1-d69c-4dd2-b7f6-5880bf1551f7","Type":"ContainerStarted","Data":"59b4e08b3189ba3020d2900961e1de77a72af71cf7ebc273a2938270917767bd"} Oct 30 18:37:33.438315 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:33.438291 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq" event={"ID":"13bb697e-8bff-4a30-9e1d-b9f3baac416e","Type":"ContainerStarted","Data":"ab324550baf6d4c95d87cb945e2000659647cc9a310fb6886a8103b790ebd1b7"} Oct 30 18:37:33.553206 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:33.553151 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-q9rgp" podStartSLOduration=312.636264612 podStartE2EDuration="5m17.553134057s" totalImagesPullingTime="4.916869445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:32:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:33.551228123 +0000 UTC m=+37.861774020" watchObservedRunningTime="2026-10-30 18:37:33.553134057 +0000 UTC m=+37.863679959" Oct 30 18:37:33.788214 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:33.788147 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6" podStartSLOduration=319.689185758 podStartE2EDuration="5m24.788127242s" totalImagesPullingTime="5.098941484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:32:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:33.668195684 +0000 UTC m=+37.978741584" watchObservedRunningTime="2026-10-30 18:37:33.788127242 +0000 UTC m=+38.098673143" Oct 30 18:37:33.908730 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:33.908680 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq" podStartSLOduration=313.452251365 podStartE2EDuration="5m18.908663825s" totalImagesPullingTime="5.45641246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:32:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:33.907883074 +0000 UTC m=+38.218428972" watchObservedRunningTime="2026-10-30 18:37:33.908663825 +0000 UTC m=+38.219209734" Oct 30 18:37:34.199445 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.199414 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bscj4"] Oct 30 18:37:34.220308 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.220284 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:34.267268 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.267221 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bscj4"] Oct 30 18:37:34.272983 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.272941 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 18:37:34.273396 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.273377 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f6d2v\"" Oct 30 18:37:34.273914 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.273897 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 18:37:34.345649 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.345316 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-data-volume\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:34.345649 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.345365 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:34.345649 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.345412 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:34.345649 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.345457 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-crio-socket\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:34.345649 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.345530 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbxh5\" (UniqueName: \"kubernetes.io/projected/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-kube-api-access-nbxh5\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:34.447349 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.447282 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-lnkcs-v7pdj"] Oct 30 18:37:34.450638 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.448931 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-dl96p\" (UID: \"1340fb2d-5f77-4316-a156-f82e0cb1124d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" Oct 30 18:37:34.450638 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.449016 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-data-volume\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:34.450638 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.449058 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:34.450638 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.449107 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:34.450638 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.449157 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-crio-socket\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:34.450638 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.449208 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbxh5\" (UniqueName: \"kubernetes.io/projected/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-kube-api-access-nbxh5\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:34.450638 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.449599 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:37:34.450638 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.449655 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert podName:1340fb2d-5f77-4316-a156-f82e0cb1124d nodeName:}" failed. No retries permitted until 2026-10-30 18:37:42.449638687 +0000 UTC m=+46.760184578 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-dl96p" (UID: "1340fb2d-5f77-4316-a156-f82e0cb1124d") : secret "networking-console-plugin-cert" not found Oct 30 18:37:34.450638 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.450221 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-data-volume\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:34.450638 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.450304 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 18:37:34.450638 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.450345 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-kube-rbac-proxy-cm podName:3ccc9d41-f2e7-4b0d-a064-cb3e5549049a nodeName:}" failed. No retries permitted until 2026-10-30 18:37:34.950331057 +0000 UTC m=+39.260876950 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bscj4" (UID: "3ccc9d41-f2e7-4b0d-a064-cb3e5549049a") : configmap "kube-rbac-proxy" not found Oct 30 18:37:34.450638 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.450412 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 18:37:34.450638 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.450443 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls podName:3ccc9d41-f2e7-4b0d-a064-cb3e5549049a nodeName:}" failed. No retries permitted until 2026-10-30 18:37:34.950432994 +0000 UTC m=+39.260978883 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bscj4" (UID: "3ccc9d41-f2e7-4b0d-a064-cb3e5549049a") : secret "insights-runtime-extractor-tls" not found Oct 30 18:37:34.450638 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.450525 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-crio-socket\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:34.451969 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.451943 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 18:37:34.453670 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.452781 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/0.log" Oct 30 18:37:34.453670 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.452817 2581 generic.go:356] "Generic (PLEG): container finished" podID="0c876fc3-6980-4a99-aed5-7439ca3d6b69" containerID="5b64ecda12ea0b0a79e9aae4b5a3057c653329e1c1e98c72fb2411e0caca677e" exitCode=255 Oct 30 18:37:34.453670 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.453231 2581 scope.go:122] "RemoveContainer" containerID="5b64ecda12ea0b0a79e9aae4b5a3057c653329e1c1e98c72fb2411e0caca677e" Oct 30 18:37:34.453670 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.453440 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-tp7sc_openshift-console-operator(0c876fc3-6980-4a99-aed5-7439ca3d6b69)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" podUID="0c876fc3-6980-4a99-aed5-7439ca3d6b69" Oct 30 18:37:34.463479 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.463454 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" event={"ID":"0c876fc3-6980-4a99-aed5-7439ca3d6b69","Type":"ContainerDied","Data":"5b64ecda12ea0b0a79e9aae4b5a3057c653329e1c1e98c72fb2411e0caca677e"} Oct 30 18:37:34.463591 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.463495 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rbmpn" event={"ID":"542d052a-724f-4d98-9be7-7765a8c1d80a","Type":"ContainerStarted","Data":"b5797c9b8c54d7bd5e892ae138e8c3dafb2cd4d1d5985b69c032ccf0ce8b67e6"} Oct 30 18:37:34.463591 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.463517 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-d9fjm" event={"ID":"68788658-4afb-4757-acfa-7638cfb2488b","Type":"ContainerStarted","Data":"36e55587ec3f7718de53e71c2adc1bbd47f744a78213919329196ba4c1e6fed8"} Oct 30 18:37:34.463591 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.463539 2581 scope.go:122] "RemoveContainer" containerID="e24b247eaedd205eb67d2516feac687e8dc32c329c6fcad6ac4dd7d07dc8a821" Oct 30 18:37:34.463591 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.463580 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:37:34.507524 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.507492 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-lnkcs-v7pdj"] Oct 30 18:37:34.513441 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.513408 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbxh5\" (UniqueName: \"kubernetes.io/projected/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-kube-api-access-nbxh5\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:34.550133 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.550105 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:34.550268 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.550153 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:34.550268 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.550235 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:34.550383 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.550271 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:34.550383 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.550302 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-metrics-certs\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:34.550383 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.550338 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:34.550383 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.550271 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:37:34.550383 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.550361 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649485556b-7x5gc: secret "image-registry-tls" not found Oct 30 18:37:34.550383 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.550363 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle podName:8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:42.550343682 +0000 UTC m=+46.860889572 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle") pod "router-default-6c975679c5-x5lg8" (UID: "8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7") : configmap references non-existent config key: service-ca.crt Oct 30 18:37:34.550383 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.550377 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:37:34.550628 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.550405 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls podName:d9b465b9-97b6-47e2-8732-ed29b8f24ed0 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:42.550391958 +0000 UTC m=+46.860937837 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls") pod "image-registry-649485556b-7x5gc" (UID: "d9b465b9-97b6-47e2-8732-ed29b8f24ed0") : secret "image-registry-tls" not found Oct 30 18:37:34.550628 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.550438 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 18:37:34.550628 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.550454 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:34.550628 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.550462 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-metrics-certs podName:8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:42.55045246 +0000 UTC m=+46.860998334 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-metrics-certs") pod "router-default-6c975679c5-x5lg8" (UID: "8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7") : secret "router-metrics-certs-default" not found Oct 30 18:37:34.550628 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.550477 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:37:34.550628 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.550489 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5df4b54dc9-8pdtm: secret "image-registry-tls" not found Oct 30 18:37:34.550628 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.550339 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:37:34.550628 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.550524 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls podName:7fcdea8f-3b0f-42c5-bb3d-b2266b046491 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:42.550510835 +0000 UTC m=+46.861056716 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls") pod "image-registry-5df4b54dc9-8pdtm" (UID: "7fcdea8f-3b0f-42c5-bb3d-b2266b046491") : secret "image-registry-tls" not found Oct 30 18:37:34.550628 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.550525 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:37:34.550628 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.550541 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls podName:447f2b64-f13e-48cc-94b9-e80df4a9d4f3 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:42.550530194 +0000 UTC m=+46.861076071 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls") pod "dns-default-ltkzx" (UID: "447f2b64-f13e-48cc-94b9-e80df4a9d4f3") : secret "dns-default-metrics-tls" not found Oct 30 18:37:34.550628 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.550556 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls podName:74153c90-cd72-43b8-85c5-aadb4d66a3e4 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:42.550547851 +0000 UTC m=+46.861093731 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-cf8qt" (UID: "74153c90-cd72-43b8-85c5-aadb4d66a3e4") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:37:34.550628 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.550570 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls podName:b18682b8-f91a-4c27-87eb-82720ba4257f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:42.550561536 +0000 UTC m=+46.861107417 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6mt5b" (UID: "b18682b8-f91a-4c27-87eb-82720ba4257f") : secret "samples-operator-tls" not found Oct 30 18:37:34.577862 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.576677 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-7kvml"] Oct 30 18:37:34.589923 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.589900 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-7kvml" Oct 30 18:37:34.608362 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.608338 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-tsbpt"] Oct 30 18:37:34.620243 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.620222 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-tsbpt" Oct 30 18:37:34.635756 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.635735 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-7kvml"] Oct 30 18:37:34.644689 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.644671 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Oct 30 18:37:34.645026 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.644958 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Oct 30 18:37:34.645663 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.645628 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-t9ljp\"" Oct 30 18:37:34.651210 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.651186 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/03157316-915c-412d-9a93-5edcf607a91f-archives-path\") pod \"periodic-gathering-lnkcs-v7pdj\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:37:34.651292 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.651223 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03157316-915c-412d-9a93-5edcf607a91f-service-ca-bundle\") pod \"periodic-gathering-lnkcs-v7pdj\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:37:34.651359 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.651322 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4bc7\" (UniqueName: \"kubernetes.io/projected/03157316-915c-412d-9a93-5edcf607a91f-kube-api-access-g4bc7\") pod \"periodic-gathering-lnkcs-v7pdj\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:37:34.651608 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.651578 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03157316-915c-412d-9a93-5edcf607a91f-trusted-ca-bundle\") pod \"periodic-gathering-lnkcs-v7pdj\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:37:34.675513 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.675488 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-tsbpt"] Oct 30 18:37:34.752638 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.752568 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mpj6\" (UniqueName: \"kubernetes.io/projected/85fa03e6-458a-45e7-8c7b-a6820618a18a-kube-api-access-2mpj6\") pod \"migrator-66df5d496b-tsbpt\" (UID: \"85fa03e6-458a-45e7-8c7b-a6820618a18a\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-tsbpt" Oct 30 18:37:34.752775 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.752693 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03157316-915c-412d-9a93-5edcf607a91f-trusted-ca-bundle\") pod \"periodic-gathering-lnkcs-v7pdj\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:37:34.752840 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.752783 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z9kz\" (UniqueName: \"kubernetes.io/projected/b4a1d050-5a83-445c-a5a9-03ce4c64cc94-kube-api-access-5z9kz\") pod \"migrator-66df5d496b-7kvml\" (UID: \"b4a1d050-5a83-445c-a5a9-03ce4c64cc94\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-7kvml" Oct 30 18:37:34.752955 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.752937 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/03157316-915c-412d-9a93-5edcf607a91f-archives-path\") pod \"periodic-gathering-lnkcs-v7pdj\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:37:34.753044 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.752967 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03157316-915c-412d-9a93-5edcf607a91f-service-ca-bundle\") pod \"periodic-gathering-lnkcs-v7pdj\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:37:34.753044 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.752998 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4bc7\" (UniqueName: \"kubernetes.io/projected/03157316-915c-412d-9a93-5edcf607a91f-kube-api-access-g4bc7\") pod \"periodic-gathering-lnkcs-v7pdj\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:37:34.753498 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.753475 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03157316-915c-412d-9a93-5edcf607a91f-trusted-ca-bundle\") pod \"periodic-gathering-lnkcs-v7pdj\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:37:34.753604 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.753508 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/03157316-915c-412d-9a93-5edcf607a91f-archives-path\") pod \"periodic-gathering-lnkcs-v7pdj\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:37:34.753877 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.753858 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03157316-915c-412d-9a93-5edcf607a91f-service-ca-bundle\") pod \"periodic-gathering-lnkcs-v7pdj\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:37:34.813249 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.813221 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4bc7\" (UniqueName: \"kubernetes.io/projected/03157316-915c-412d-9a93-5edcf607a91f-kube-api-access-g4bc7\") pod \"periodic-gathering-lnkcs-v7pdj\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:37:34.853528 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.853494 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mpj6\" (UniqueName: \"kubernetes.io/projected/85fa03e6-458a-45e7-8c7b-a6820618a18a-kube-api-access-2mpj6\") pod \"migrator-66df5d496b-tsbpt\" (UID: \"85fa03e6-458a-45e7-8c7b-a6820618a18a\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-tsbpt" Oct 30 18:37:34.853656 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.853619 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z9kz\" (UniqueName: \"kubernetes.io/projected/b4a1d050-5a83-445c-a5a9-03ce4c64cc94-kube-api-access-5z9kz\") pod \"migrator-66df5d496b-7kvml\" (UID: \"b4a1d050-5a83-445c-a5a9-03ce4c64cc94\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-7kvml" Oct 30 18:37:34.913306 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.913277 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mpj6\" (UniqueName: \"kubernetes.io/projected/85fa03e6-458a-45e7-8c7b-a6820618a18a-kube-api-access-2mpj6\") pod \"migrator-66df5d496b-tsbpt\" (UID: \"85fa03e6-458a-45e7-8c7b-a6820618a18a\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-tsbpt" Oct 30 18:37:34.915393 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.915372 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z9kz\" (UniqueName: \"kubernetes.io/projected/b4a1d050-5a83-445c-a5a9-03ce4c64cc94-kube-api-access-5z9kz\") pod \"migrator-66df5d496b-7kvml\" (UID: \"b4a1d050-5a83-445c-a5a9-03ce4c64cc94\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-7kvml" Oct 30 18:37:34.916053 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.916028 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-7kvml" Oct 30 18:37:34.942732 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.942559 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-tsbpt" Oct 30 18:37:34.954603 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.954481 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-d9fjm" podStartSLOduration=311.674940152 podStartE2EDuration="5m17.954462693s" totalImagesPullingTime="6.279522541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:32:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:34.812821376 +0000 UTC m=+39.123367287" watchObservedRunningTime="2026-10-30 18:37:34.954462693 +0000 UTC m=+39.265008592" Oct 30 18:37:34.955020 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.954726 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 18:37:34.955619 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.955513 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:34.955619 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:34.955576 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:34.955889 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.955862 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-kube-rbac-proxy-cm podName:3ccc9d41-f2e7-4b0d-a064-cb3e5549049a nodeName:}" failed. No retries permitted until 2026-10-30 18:37:35.955841533 +0000 UTC m=+40.266387408 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bscj4" (UID: "3ccc9d41-f2e7-4b0d-a064-cb3e5549049a") : configmap "kube-rbac-proxy" not found Oct 30 18:37:34.955997 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.955889 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 18:37:34.955997 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:34.955935 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls podName:3ccc9d41-f2e7-4b0d-a064-cb3e5549049a nodeName:}" failed. No retries permitted until 2026-10-30 18:37:35.955921999 +0000 UTC m=+40.266467874 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bscj4" (UID: "3ccc9d41-f2e7-4b0d-a064-cb3e5549049a") : secret "insights-runtime-extractor-tls" not found Oct 30 18:37:35.095569 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.095515 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rbmpn" podStartSLOduration=3.087225357 podStartE2EDuration="9.095500597s" totalImagesPullingTime="6.00827524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:34.954216375 +0000 UTC m=+39.264762283" watchObservedRunningTime="2026-10-30 18:37:35.095500597 +0000 UTC m=+39.406046494" Oct 30 18:37:35.095744 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.095722 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:37:35.308587 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.308508 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-7kvml"] Oct 30 18:37:35.313904 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:37:35.313869 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4a1d050_5a83_445c_a5a9_03ce4c64cc94.slice/crio-52675bff9e6f9c76c989e6670a662df85a4741136b9da946053cc2102dc2f54b WatchSource:0}: Error finding container 52675bff9e6f9c76c989e6670a662df85a4741136b9da946053cc2102dc2f54b: Status 404 returned error can't find the container with id 52675bff9e6f9c76c989e6670a662df85a4741136b9da946053cc2102dc2f54b Oct 30 18:37:35.377070 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.377041 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-tsbpt"] Oct 30 18:37:35.383324 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:37:35.383292 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85fa03e6_458a_45e7_8c7b_a6820618a18a.slice/crio-25485677ed922f926f62f71fd46ef2ee95a205b5e3ed9bd16dfe9ad3a7d07714 WatchSource:0}: Error finding container 25485677ed922f926f62f71fd46ef2ee95a205b5e3ed9bd16dfe9ad3a7d07714: Status 404 returned error can't find the container with id 25485677ed922f926f62f71fd46ef2ee95a205b5e3ed9bd16dfe9ad3a7d07714 Oct 30 18:37:35.400266 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.400245 2581 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-rdt4m"] Oct 30 18:37:35.424793 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.424774 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-rdt4m" Oct 30 18:37:35.460909 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.460861 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-rdt4m"] Oct 30 18:37:35.467284 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.467254 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-62vfn" event={"ID":"97294b18-463b-429d-8073-dc532ab0a046","Type":"ContainerStarted","Data":"2b71229dcac90ce8c8ba14571c05542385d613fcfeada195b8628c97ec990dd2"} Oct 30 18:37:35.467394 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.467322 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:35.469120 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.469100 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 18:37:35.469497 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.469481 2581 scope.go:122] "RemoveContainer" containerID="5b64ecda12ea0b0a79e9aae4b5a3057c653329e1c1e98c72fb2411e0caca677e" Oct 30 18:37:35.469659 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:35.469637 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-tp7sc_openshift-console-operator(0c876fc3-6980-4a99-aed5-7439ca3d6b69)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" podUID="0c876fc3-6980-4a99-aed5-7439ca3d6b69" Oct 30 18:37:35.470748 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.470725 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-tsbpt" event={"ID":"85fa03e6-458a-45e7-8c7b-a6820618a18a","Type":"ContainerStarted","Data":"25485677ed922f926f62f71fd46ef2ee95a205b5e3ed9bd16dfe9ad3a7d07714"} Oct 30 18:37:35.471919 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.471894 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-7kvml" event={"ID":"b4a1d050-5a83-445c-a5a9-03ce4c64cc94","Type":"ContainerStarted","Data":"52675bff9e6f9c76c989e6670a662df85a4741136b9da946053cc2102dc2f54b"} Oct 30 18:37:35.479101 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.479079 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-q6kg8\"" Oct 30 18:37:35.479192 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.479155 2581 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Oct 30 18:37:35.479772 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.479757 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Oct 30 18:37:35.479841 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.479773 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Oct 30 18:37:35.479886 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.479877 2581 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Oct 30 18:37:35.495422 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.495404 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-lnkcs-v7pdj"] Oct 30 18:37:35.501776 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:37:35.501753 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03157316_915c_412d_9a93_5edcf607a91f.slice/crio-53a0a315d1ae630f72c402a412e936660f22e80450846e2c9d0a0a05ecf09d35 WatchSource:0}: Error finding container 53a0a315d1ae630f72c402a412e936660f22e80450846e2c9d0a0a05ecf09d35: Status 404 returned error can't find the container with id 53a0a315d1ae630f72c402a412e936660f22e80450846e2c9d0a0a05ecf09d35 Oct 30 18:37:35.561020 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.560972 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a74aa745-ed96-4880-ad11-a31391e40f57-signing-cabundle\") pod \"service-ca-5fffc97bdf-rdt4m\" (UID: \"a74aa745-ed96-4880-ad11-a31391e40f57\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rdt4m" Oct 30 18:37:35.561155 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.561135 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a74aa745-ed96-4880-ad11-a31391e40f57-signing-key\") pod \"service-ca-5fffc97bdf-rdt4m\" (UID: \"a74aa745-ed96-4880-ad11-a31391e40f57\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rdt4m" Oct 30 18:37:35.561229 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.561190 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5chv\" (UniqueName: \"kubernetes.io/projected/a74aa745-ed96-4880-ad11-a31391e40f57-kube-api-access-k5chv\") pod \"service-ca-5fffc97bdf-rdt4m\" (UID: \"a74aa745-ed96-4880-ad11-a31391e40f57\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rdt4m" Oct 30 18:37:35.650333 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.650269 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-62vfn" podStartSLOduration=35.633163921 podStartE2EDuration="39.650251834s" totalImagesPullingTime="4.017087913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:35.648830405 +0000 UTC m=+39.959376327" watchObservedRunningTime="2026-10-30 18:37:35.650251834 +0000 UTC m=+39.960797733" Oct 30 18:37:35.663322 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.661730 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a74aa745-ed96-4880-ad11-a31391e40f57-signing-cabundle\") pod \"service-ca-5fffc97bdf-rdt4m\" (UID: \"a74aa745-ed96-4880-ad11-a31391e40f57\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rdt4m" Oct 30 18:37:35.663322 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.661885 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a74aa745-ed96-4880-ad11-a31391e40f57-signing-key\") pod \"service-ca-5fffc97bdf-rdt4m\" (UID: \"a74aa745-ed96-4880-ad11-a31391e40f57\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rdt4m" Oct 30 18:37:35.663322 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.662030 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5chv\" (UniqueName: \"kubernetes.io/projected/a74aa745-ed96-4880-ad11-a31391e40f57-kube-api-access-k5chv\") pod \"service-ca-5fffc97bdf-rdt4m\" (UID: \"a74aa745-ed96-4880-ad11-a31391e40f57\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rdt4m" Oct 30 18:37:35.663322 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.662540 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a74aa745-ed96-4880-ad11-a31391e40f57-signing-cabundle\") pod \"service-ca-5fffc97bdf-rdt4m\" (UID: \"a74aa745-ed96-4880-ad11-a31391e40f57\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rdt4m" Oct 30 18:37:35.664771 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.664727 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a74aa745-ed96-4880-ad11-a31391e40f57-signing-key\") pod \"service-ca-5fffc97bdf-rdt4m\" (UID: \"a74aa745-ed96-4880-ad11-a31391e40f57\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rdt4m" Oct 30 18:37:35.723458 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.723430 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5chv\" (UniqueName: \"kubernetes.io/projected/a74aa745-ed96-4880-ad11-a31391e40f57-kube-api-access-k5chv\") pod \"service-ca-5fffc97bdf-rdt4m\" (UID: \"a74aa745-ed96-4880-ad11-a31391e40f57\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rdt4m" Oct 30 18:37:35.753575 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.753550 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-rdt4m" Oct 30 18:37:35.965024 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.964974 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:35.965183 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:35.965079 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:35.965183 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:35.965111 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 18:37:35.965183 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:35.965177 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-kube-rbac-proxy-cm podName:3ccc9d41-f2e7-4b0d-a064-cb3e5549049a nodeName:}" failed. No retries permitted until 2026-10-30 18:37:37.965162731 +0000 UTC m=+42.275708605 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bscj4" (UID: "3ccc9d41-f2e7-4b0d-a064-cb3e5549049a") : configmap "kube-rbac-proxy" not found Oct 30 18:37:35.965339 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:35.965183 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 18:37:35.965339 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:35.965235 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls podName:3ccc9d41-f2e7-4b0d-a064-cb3e5549049a nodeName:}" failed. No retries permitted until 2026-10-30 18:37:37.965218526 +0000 UTC m=+42.275764419 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bscj4" (UID: "3ccc9d41-f2e7-4b0d-a064-cb3e5549049a") : secret "insights-runtime-extractor-tls" not found Oct 30 18:37:36.146603 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:36.146571 2581 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-rdt4m"] Oct 30 18:37:36.152483 ip-10-0-132-74 kubenswrapper[2581]: W1030 18:37:36.152454 2581 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda74aa745_ed96_4880_ad11_a31391e40f57.slice/crio-fb54c0a85ed5f3cc90ad6ef821488242fedb80ea7320f94810dbf3bf32ab10ea WatchSource:0}: Error finding container fb54c0a85ed5f3cc90ad6ef821488242fedb80ea7320f94810dbf3bf32ab10ea: Status 404 returned error can't find the container with id fb54c0a85ed5f3cc90ad6ef821488242fedb80ea7320f94810dbf3bf32ab10ea Oct 30 18:37:36.481517 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:36.480752 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-rdt4m" event={"ID":"a74aa745-ed96-4880-ad11-a31391e40f57","Type":"ContainerStarted","Data":"79280dc4f03028c0c0cd042e0bd04fd1d955a3c5ceb864e290d31d48b9a18e1a"} Oct 30 18:37:36.481517 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:36.480809 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-rdt4m" event={"ID":"a74aa745-ed96-4880-ad11-a31391e40f57","Type":"ContainerStarted","Data":"fb54c0a85ed5f3cc90ad6ef821488242fedb80ea7320f94810dbf3bf32ab10ea"} Oct 30 18:37:36.484086 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:36.484047 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" event={"ID":"03157316-915c-412d-9a93-5edcf607a91f","Type":"ContainerStarted","Data":"cb2f4e8848a187011f38af6c5bc17c6f96a82ab02636a0a8511074a7a330fffd"} Oct 30 18:37:36.484216 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:36.484091 2581 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" event={"ID":"03157316-915c-412d-9a93-5edcf607a91f","Type":"ContainerStarted","Data":"53a0a315d1ae630f72c402a412e936660f22e80450846e2c9d0a0a05ecf09d35"} Oct 30 18:37:36.597890 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:36.597217 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-rdt4m" podStartSLOduration=1.5972029669999999 podStartE2EDuration="1.597202967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:36.597187075 +0000 UTC m=+40.907732974" watchObservedRunningTime="2026-10-30 18:37:36.597202967 +0000 UTC m=+40.907748864" Oct 30 18:37:36.711167 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:36.711116 2581 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" podStartSLOduration=2.7111040429999997 podStartE2EDuration="2.711104043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:36.71061182 +0000 UTC m=+41.021157719" watchObservedRunningTime="2026-10-30 18:37:36.711104043 +0000 UTC m=+41.021649938" Oct 30 18:37:37.074520 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:37.074483 2581 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:37.074692 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:37.074561 2581 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:37.075012 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:37.074978 2581 scope.go:122] "RemoveContainer" containerID="5b64ecda12ea0b0a79e9aae4b5a3057c653329e1c1e98c72fb2411e0caca677e" Oct 30 18:37:37.075253 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:37.075231 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-tp7sc_openshift-console-operator(0c876fc3-6980-4a99-aed5-7439ca3d6b69)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" podUID="0c876fc3-6980-4a99-aed5-7439ca3d6b69" Oct 30 18:37:37.992130 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:37.991754 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:37.992130 ip-10-0-132-74 kubenswrapper[2581]: I1030 18:37:37.991820 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:37.992657 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:37.992632 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 18:37:37.992774 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:37.992701 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls podName:3ccc9d41-f2e7-4b0d-a064-cb3e5549049a nodeName:}" failed. No retries permitted until 2026-10-30 18:37:41.992680594 +0000 UTC m=+46.303226486 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bscj4" (UID: "3ccc9d41-f2e7-4b0d-a064-cb3e5549049a") : secret "insights-runtime-extractor-tls" not found Oct 30 18:37:37.992774 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:37.992753 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 18:37:37.992857 ip-10-0-132-74 kubenswrapper[2581]: E1030 18:37:37.992785 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-kube-rbac-proxy-cm podName:3ccc9d41-f2e7-4b0d-a064-cb3e5549049a nodeName:}" failed. No retries permitted until 2026-10-30 18:37:41.992774184 +0000 UTC m=+46.303320073 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bscj4" (UID: "3ccc9d41-f2e7-4b0d-a064-cb3e5549049a") : configmap "kube-rbac-proxy" not found Oct 30 18:37:38.199654 ip-10-0-132-74 systemd[1]: Stopping Kubernetes Kubelet... Oct 30 18:37:38.210432 ip-10-0-132-74 systemd[1]: kubelet.service: Deactivated successfully. Oct 30 18:37:38.210647 ip-10-0-132-74 systemd[1]: Stopped Kubernetes Kubelet. Oct 30 18:37:38.210703 ip-10-0-132-74 systemd[1]: kubelet.service: Consumed 3.062s CPU time. Oct 30 18:37:38.225574 ip-10-0-132-74 systemd[1]: Starting Kubernetes Kubelet... Oct 30 18:37:38.339441 ip-10-0-132-74 kubenswrapper[5054]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:37:38.339441 ip-10-0-132-74 kubenswrapper[5054]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 18:37:38.339441 ip-10-0-132-74 kubenswrapper[5054]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:37:38.339441 ip-10-0-132-74 kubenswrapper[5054]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:37:38.341457 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341374 5054 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:38.341457 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341456 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:38.341457 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341459 5054 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:38.341457 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341464 5054 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:38.341603 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341467 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:38.341603 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341470 5054 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:38.341603 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341473 5054 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:38.341603 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341476 5054 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:38.341603 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341478 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:38.341603 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341481 5054 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:38.341603 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341484 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:38.341603 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341487 5054 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:38.341603 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341489 5054 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:38.341603 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341492 5054 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:38.341603 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341495 5054 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:38.341603 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341497 5054 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:38.341603 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341500 5054 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:38.341603 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341503 5054 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:38.341603 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341505 5054 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:38.341603 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341508 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:38.341603 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341511 5054 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:38.341603 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341514 5054 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:38.341603 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341517 5054 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:38.341603 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341520 5054 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:38.342133 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341522 5054 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:38.342133 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341525 5054 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:38.342133 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341540 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:38.342133 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341543 5054 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:38.342133 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341545 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:38.342133 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341550 5054 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:38.342133 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341554 5054 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:38.342133 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341557 5054 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:38.342133 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341560 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:38.342133 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341563 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:38.342133 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341566 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:38.342133 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341568 5054 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:38.342133 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341572 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:38.342133 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341574 5054 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:38.342133 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341577 5054 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:38.342133 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341580 5054 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:38.342133 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341583 5054 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:38.342133 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341586 5054 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:38.342133 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341589 5054 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:38.342590 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341591 5054 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:38.342590 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341594 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:38.342590 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341596 5054 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:38.342590 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341599 5054 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:38.342590 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341602 5054 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:38.342590 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341604 5054 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:38.342590 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341607 5054 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:38.342590 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341610 5054 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:38.342590 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341614 5054 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:38.342590 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341620 5054 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:38.342590 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341624 5054 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:38.342590 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341629 5054 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:38.342590 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341634 5054 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:38.342590 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341638 5054 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:38.342590 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341642 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:38.342590 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341646 5054 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:38.342590 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341650 5054 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:38.342590 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341655 5054 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:38.342590 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341659 5054 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:38.342590 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341663 5054 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:38.343165 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341668 5054 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:38.343165 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341671 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:38.343165 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341674 5054 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:38.343165 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341677 5054 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:38.343165 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341680 5054 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:38.343165 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341682 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:38.343165 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341685 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:38.343165 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341688 5054 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:38.343165 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341691 5054 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:38.343165 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341693 5054 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:38.343165 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341696 5054 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:38.343165 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341698 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:38.343165 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341701 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:38.343165 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341704 5054 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:38.343165 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341706 5054 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:38.343165 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341710 5054 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:38.343165 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341714 5054 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:38.343165 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341717 5054 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:38.343165 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341720 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:38.343666 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341724 5054 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:38.343666 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341727 5054 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:38.343666 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341730 5054 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:38.343666 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341733 5054 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:38.343666 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341735 5054 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:38.343666 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341739 5054 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:38.343666 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341742 5054 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:38.343666 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341745 5054 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:38.343666 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341747 5054 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:38.343666 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341750 5054 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:38.343666 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341752 5054 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:38.343666 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341755 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:38.343666 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341757 5054 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:38.343666 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341760 5054 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:38.343666 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341763 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:38.343666 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341766 5054 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:38.343666 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341768 5054 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:38.343666 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341771 5054 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:38.343666 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341774 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:38.343666 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341776 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:38.344199 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341779 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:38.344199 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341782 5054 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:38.344199 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341784 5054 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:38.344199 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.341788 5054 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:38.344199 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342574 5054 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:38.344199 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342582 5054 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:38.344199 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342585 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:38.344199 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342588 5054 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:38.344199 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342591 5054 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:38.344199 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342594 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:38.344199 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342597 5054 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:38.344199 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342599 5054 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:38.344199 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342603 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:38.344199 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342606 5054 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:38.344199 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342609 5054 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:38.344199 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342611 5054 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:38.344199 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342614 5054 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:38.344199 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342617 5054 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:38.344199 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342620 5054 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:38.344672 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342623 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:38.344672 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342626 5054 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:38.344672 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342629 5054 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:38.344672 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342631 5054 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:38.344672 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342634 5054 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:38.344672 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342637 5054 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:38.344672 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342639 5054 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:38.344672 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342642 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:38.344672 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342644 5054 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:38.344672 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342649 5054 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:38.344672 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342653 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:38.344672 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342656 5054 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:38.344672 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342659 5054 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:38.344672 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342662 5054 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:38.344672 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342665 5054 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:38.344672 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342667 5054 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:38.344672 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342671 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:38.344672 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342674 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:38.344672 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342676 5054 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:38.345179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342680 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:38.345179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342683 5054 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:38.345179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342685 5054 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:38.345179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342688 5054 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:38.345179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342691 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:38.345179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342693 5054 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:38.345179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342696 5054 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:38.345179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342699 5054 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:38.345179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342702 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:38.345179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342704 5054 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:38.345179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342707 5054 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:38.345179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342709 5054 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:38.345179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342712 5054 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:38.345179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342715 5054 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:38.345179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342718 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:38.345179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342720 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:38.345179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342723 5054 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:38.345179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342725 5054 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:38.345179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342730 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:38.345179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342734 5054 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:38.345708 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342737 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:38.345708 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342742 5054 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:38.345708 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342746 5054 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:38.345708 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342750 5054 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:38.345708 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342754 5054 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:38.345708 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342759 5054 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:38.345708 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342763 5054 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:38.345708 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342766 5054 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:38.345708 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342769 5054 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:38.345708 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342771 5054 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:38.345708 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342774 5054 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:38.345708 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342777 5054 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:38.345708 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342779 5054 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:38.345708 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342782 5054 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:38.345708 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342785 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:38.345708 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342787 5054 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:38.345708 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342790 5054 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:38.345708 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342792 5054 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:38.345708 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342796 5054 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:38.345708 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342798 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:38.346250 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342801 5054 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:38.346250 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342804 5054 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:38.346250 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342806 5054 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:38.346250 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342809 5054 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:38.346250 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342811 5054 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:38.346250 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342814 5054 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:38.346250 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342817 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:38.346250 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342820 5054 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:38.346250 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342823 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:38.346250 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342825 5054 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:38.346250 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342828 5054 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:38.346250 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342830 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:38.346250 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342833 5054 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:38.346250 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342835 5054 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:38.346250 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342841 5054 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:38.346250 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342844 5054 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:38.346250 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342846 5054 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:38.346250 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342849 5054 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:38.346250 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342851 5054 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:38.346250 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342854 5054 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:38.346780 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342856 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:38.346780 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342859 5054 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:38.346780 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342861 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:38.346780 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342864 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:38.346780 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342867 5054 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:38.346780 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342869 5054 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:38.346780 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342872 5054 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:38.346780 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342874 5054 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:38.346780 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342879 5054 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:38.346780 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342883 5054 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:38.346780 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342887 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:38.346780 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.342890 5054 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:38.346780 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343398 5054 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:38.346780 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343404 5054 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:38.346780 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343408 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:38.346780 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343410 5054 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:38.346780 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343413 5054 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:38.346780 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343415 5054 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:38.346780 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343424 5054 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:38.347277 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343429 5054 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:38.347277 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343433 5054 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:38.347277 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343437 5054 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:38.347277 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343440 5054 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:38.347277 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343442 5054 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:38.347277 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343445 5054 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:38.347277 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343447 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:38.347277 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343452 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:38.347277 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343455 5054 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:38.347277 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343457 5054 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:38.347277 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343460 5054 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:38.347277 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343463 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:38.347277 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343465 5054 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:38.347277 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343468 5054 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:38.347277 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343470 5054 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:38.347277 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343473 5054 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:38.347277 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343476 5054 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:38.347277 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343478 5054 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:38.347277 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343481 5054 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:38.347763 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343483 5054 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:38.347763 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343486 5054 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:38.347763 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343489 5054 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:38.347763 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343491 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:38.347763 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343494 5054 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:38.347763 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343497 5054 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:38.347763 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343499 5054 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:38.347763 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343502 5054 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:38.347763 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343504 5054 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:38.347763 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343507 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:38.347763 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343510 5054 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:38.347763 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343512 5054 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:38.347763 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343515 5054 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:38.347763 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343518 5054 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:38.347763 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343520 5054 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:38.347763 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343523 5054 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:38.347763 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343526 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:38.347763 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343528 5054 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:38.347763 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343531 5054 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:38.347763 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343533 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:38.348323 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343538 5054 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:38.348323 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343540 5054 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:38.348323 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343543 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:38.348323 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343545 5054 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:38.348323 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343548 5054 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:38.348323 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343550 5054 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:38.348323 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343553 5054 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:38.348323 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343555 5054 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:38.348323 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343558 5054 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:38.348323 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343561 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:38.348323 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343564 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:38.348323 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343566 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:38.348323 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343569 5054 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:38.348323 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343572 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:38.348323 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343575 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:38.348323 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343577 5054 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:38.348323 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343580 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:38.348323 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343582 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:38.348323 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343585 5054 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:38.348793 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343588 5054 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:38.348793 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343590 5054 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:38.348793 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343593 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:38.348793 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343597 5054 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:38.348793 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343601 5054 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:38.348793 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343604 5054 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:38.348793 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343606 5054 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:38.348793 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343609 5054 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:38.348793 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343612 5054 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:38.348793 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343614 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:38.348793 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343617 5054 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:38.348793 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343620 5054 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:38.348793 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343622 5054 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:38.348793 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343627 5054 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:38.348793 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343630 5054 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:38.348793 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343633 5054 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:38.348793 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343635 5054 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:38.348793 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343638 5054 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:38.348793 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343641 5054 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:38.349268 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343643 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:38.349268 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343646 5054 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:38.349268 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343649 5054 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:38.349268 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343651 5054 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:38.349268 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343654 5054 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:38.349268 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343656 5054 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:38.349268 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343659 5054 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:38.349268 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343661 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:38.349268 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343664 5054 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:38.349268 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343668 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:38.349268 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343671 5054 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:38.349268 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343674 5054 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:38.349268 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343676 5054 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:38.349268 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343679 5054 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:38.349268 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343682 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:38.349268 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343684 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:38.349268 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343687 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:38.349268 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343690 5054 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:38.349268 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343692 5054 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:38.349268 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343695 5054 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343697 5054 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.343701 5054 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343797 5054 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343804 5054 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343810 5054 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343814 5054 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343821 5054 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343824 5054 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343829 5054 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343833 5054 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343836 5054 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343839 5054 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343843 5054 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343846 5054 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343850 5054 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343853 5054 flags.go:64] FLAG: --cgroup-root="" Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343856 5054 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343859 5054 flags.go:64] FLAG: --client-ca-file="" Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343862 5054 flags.go:64] FLAG: --cloud-provider="external" Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343865 5054 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343869 5054 flags.go:64] FLAG: --cluster-domain="" Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343873 5054 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343876 5054 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 18:37:38.349841 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343880 5054 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343883 5054 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343887 5054 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343890 5054 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343893 5054 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343896 5054 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343900 5054 flags.go:64] FLAG: --contention-profiling="false" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343903 5054 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343906 5054 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343909 5054 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343912 5054 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343922 5054 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343926 5054 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343929 5054 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343932 5054 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343937 5054 flags.go:64] FLAG: --enable-server="true" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343940 5054 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343944 5054 flags.go:64] FLAG: --event-burst="100" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343947 5054 flags.go:64] FLAG: --event-qps="50" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343950 5054 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343953 5054 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343956 5054 flags.go:64] FLAG: --eviction-hard="" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343960 5054 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343963 5054 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343967 5054 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 18:37:38.350440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343970 5054 flags.go:64] FLAG: --eviction-soft="" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343973 5054 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343976 5054 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343979 5054 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343982 5054 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343986 5054 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343989 5054 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343992 5054 flags.go:64] FLAG: --feature-gates="" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.343996 5054 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344013 5054 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344017 5054 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344020 5054 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344023 5054 flags.go:64] FLAG: --healthz-port="10248" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344026 5054 flags.go:64] FLAG: --help="false" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344029 5054 flags.go:64] FLAG: --hostname-override="ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344033 5054 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344036 5054 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344038 5054 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344042 5054 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344051 5054 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344055 5054 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344057 5054 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344062 5054 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 18:37:38.351290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344065 5054 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344069 5054 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344072 5054 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344075 5054 flags.go:64] FLAG: --kube-reserved="" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344078 5054 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344081 5054 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344084 5054 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344087 5054 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344089 5054 flags.go:64] FLAG: --lock-file="" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344092 5054 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344095 5054 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344098 5054 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344104 5054 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344107 5054 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344111 5054 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344114 5054 flags.go:64] FLAG: --logging-format="text" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344117 5054 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344121 5054 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344124 5054 flags.go:64] FLAG: --manifest-url="" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344126 5054 flags.go:64] FLAG: --manifest-url-header="" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344131 5054 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344134 5054 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344137 5054 flags.go:64] FLAG: --max-pods="110" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344140 5054 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344143 5054 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 18:37:38.352267 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344146 5054 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344149 5054 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344152 5054 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344155 5054 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344165 5054 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344172 5054 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344177 5054 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344180 5054 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344183 5054 flags.go:64] FLAG: --pod-cidr="" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344186 5054 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344188 5054 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344192 5054 flags.go:64] FLAG: --pods-per-core="0" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344194 5054 flags.go:64] FLAG: --port="10250" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344197 5054 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344200 5054 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-0fa259a1438c7dd7f" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344204 5054 flags.go:64] FLAG: --qos-reserved="" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344207 5054 flags.go:64] FLAG: --read-only-port="10255" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344210 5054 flags.go:64] FLAG: --register-node="true" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344213 5054 flags.go:64] FLAG: --register-with-taints="" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344217 5054 flags.go:64] FLAG: --registry-burst="10" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344220 5054 flags.go:64] FLAG: --registry-qps="5" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344224 5054 flags.go:64] FLAG: --reserved-cpus="" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344227 5054 flags.go:64] FLAG: --reserved-memory="" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344231 5054 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344234 5054 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344237 5054 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 18:37:38.353362 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344240 5054 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344243 5054 flags.go:64] FLAG: --runonce="false" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344246 5054 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344249 5054 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344252 5054 flags.go:64] FLAG: --seccomp-default="false" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344254 5054 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344257 5054 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344260 5054 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344264 5054 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344267 5054 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344269 5054 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344274 5054 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344277 5054 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344280 5054 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344284 5054 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344287 5054 flags.go:64] FLAG: --system-cgroups="" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344289 5054 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344295 5054 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344297 5054 flags.go:64] FLAG: --tls-cert-file="" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344300 5054 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344304 5054 flags.go:64] FLAG: --tls-min-version="" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344307 5054 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344310 5054 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344312 5054 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344315 5054 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 18:37:38.354546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344318 5054 flags.go:64] FLAG: --v="2" Oct 30 18:37:38.355482 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344322 5054 flags.go:64] FLAG: --version="false" Oct 30 18:37:38.355482 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344328 5054 flags.go:64] FLAG: --vmodule="" Oct 30 18:37:38.355482 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344340 5054 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 18:37:38.355482 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344344 5054 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 18:37:38.355482 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344479 5054 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:38.355482 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344483 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:38.355482 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344486 5054 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:38.355482 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344489 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:38.355482 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344491 5054 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:38.355482 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344494 5054 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:38.355482 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344497 5054 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:38.355482 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344499 5054 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:38.355482 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344502 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:38.355482 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344505 5054 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:38.355482 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344509 5054 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:38.355482 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344513 5054 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:38.355482 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344516 5054 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:38.355482 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344519 5054 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:38.355482 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344523 5054 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:38.356088 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344525 5054 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:38.356088 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344528 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:38.356088 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344531 5054 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:38.356088 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344534 5054 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:38.356088 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344536 5054 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:38.356088 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344539 5054 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:38.356088 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344541 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:38.356088 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344544 5054 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:38.356088 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344547 5054 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:38.356088 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344549 5054 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:38.356088 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344552 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:38.356088 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344554 5054 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:38.356088 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344557 5054 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:38.356088 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344560 5054 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:38.356088 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344564 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:38.356088 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344566 5054 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:38.356088 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344569 5054 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:38.356088 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344572 5054 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:38.356088 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344575 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:38.356088 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344577 5054 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:38.356687 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344580 5054 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:38.356687 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344582 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:38.356687 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344585 5054 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:38.356687 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344588 5054 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:38.356687 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344590 5054 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:38.356687 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344593 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:38.356687 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344595 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:38.356687 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344598 5054 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:38.356687 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344600 5054 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:38.356687 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344603 5054 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:38.356687 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344606 5054 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:38.356687 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344608 5054 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:38.356687 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344611 5054 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:38.356687 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344614 5054 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:38.356687 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344616 5054 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:38.356687 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344619 5054 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:38.356687 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344621 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:38.356687 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344624 5054 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:38.356687 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344626 5054 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:38.356687 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344629 5054 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:38.357215 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344631 5054 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:38.357215 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344634 5054 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:38.357215 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344636 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:38.357215 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344639 5054 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:38.357215 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344641 5054 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:38.357215 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344644 5054 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:38.357215 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344648 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:38.357215 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344651 5054 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:38.357215 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344653 5054 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:38.357215 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344656 5054 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:38.357215 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344659 5054 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:38.357215 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344661 5054 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:38.357215 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344664 5054 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:38.357215 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344666 5054 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:38.357215 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344669 5054 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:38.357215 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344671 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:38.357215 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344674 5054 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:38.357215 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344676 5054 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:38.357215 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344679 5054 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:38.357215 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344682 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:38.357752 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344684 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:38.357752 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344687 5054 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:38.357752 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344690 5054 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:38.357752 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344692 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:38.357752 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344694 5054 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:38.357752 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344697 5054 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:38.357752 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344700 5054 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:38.357752 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344703 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:38.357752 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344705 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:38.357752 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344708 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:38.357752 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344710 5054 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:38.357752 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344713 5054 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:38.357752 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344718 5054 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:38.357752 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344720 5054 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:38.357752 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344723 5054 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:38.357752 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344726 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:38.357752 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344728 5054 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:38.357752 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344731 5054 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:38.357752 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344735 5054 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:38.357752 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344737 5054 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:38.358567 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344740 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:38.358567 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344743 5054 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:38.358567 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344745 5054 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:38.358567 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344748 5054 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:38.358567 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344750 5054 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:38.358567 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344753 5054 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:38.358567 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344756 5054 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:38.358567 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344758 5054 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:38.358567 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344761 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:38.358567 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344763 5054 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:38.358567 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344767 5054 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:38.358567 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.344793 5054 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:37:38.358567 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344827 5054 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:38.358567 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344832 5054 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:38.358567 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344835 5054 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:38.358983 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344838 5054 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:38.358983 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344841 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:38.358983 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344843 5054 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:38.358983 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344846 5054 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:38.358983 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344849 5054 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:38.358983 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344852 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:38.358983 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344854 5054 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:38.358983 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344857 5054 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:38.358983 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344860 5054 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:38.358983 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344863 5054 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:38.358983 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344867 5054 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:38.358983 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344869 5054 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:38.358983 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344872 5054 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:38.358983 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344874 5054 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:38.358983 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344877 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:38.358983 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344881 5054 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:38.358983 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344884 5054 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:38.358983 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344887 5054 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:38.358983 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344889 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:38.358983 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344892 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:38.359521 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344894 5054 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:38.359521 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344897 5054 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:38.359521 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344900 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:38.359521 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344902 5054 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:38.359521 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344905 5054 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:38.359521 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344908 5054 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:38.359521 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344910 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:38.359521 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344913 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:38.359521 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344916 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:38.359521 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344918 5054 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:38.359521 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344921 5054 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:38.359521 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344923 5054 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:38.359521 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344926 5054 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:38.359521 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344929 5054 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:38.359521 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344931 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:38.359521 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344934 5054 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:38.359521 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344936 5054 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:38.359521 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344939 5054 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:38.359521 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344942 5054 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:38.359521 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344944 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:38.360157 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344947 5054 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:38.360157 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344949 5054 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:38.360157 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344954 5054 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:38.360157 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344956 5054 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:38.360157 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344959 5054 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:38.360157 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344961 5054 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:38.360157 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344964 5054 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:38.360157 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344968 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:38.360157 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344971 5054 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:38.360157 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344974 5054 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:38.360157 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344978 5054 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:38.360157 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344981 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:38.360157 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344984 5054 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:38.360157 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344987 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:38.360157 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344989 5054 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:38.360157 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344992 5054 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:38.360157 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344994 5054 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:38.360157 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.344997 5054 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:38.360157 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345012 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:38.360652 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345018 5054 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:38.360652 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345033 5054 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:38.360652 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345037 5054 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:38.360652 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345040 5054 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:38.360652 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345044 5054 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:38.360652 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345047 5054 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:38.360652 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345050 5054 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:38.360652 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345052 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:38.360652 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345055 5054 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:38.360652 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345057 5054 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:38.360652 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345060 5054 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:38.360652 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345063 5054 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:38.360652 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345066 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:38.360652 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345068 5054 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:38.360652 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345071 5054 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:38.360652 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345075 5054 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:38.360652 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345078 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:38.360652 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345081 5054 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:38.360652 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345083 5054 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:38.361179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345086 5054 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:38.361179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345089 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:38.361179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345092 5054 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:38.361179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345095 5054 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:38.361179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345098 5054 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:38.361179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345100 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:38.361179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345103 5054 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:38.361179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345105 5054 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:38.361179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345108 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:38.361179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345111 5054 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:38.361179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345114 5054 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:38.361179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345116 5054 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:38.361179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345119 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:38.361179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345121 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:38.361179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345124 5054 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:38.361179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345127 5054 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:38.361179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345130 5054 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:38.361179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345133 5054 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:38.361179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345135 5054 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:38.361179 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345138 5054 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:38.361699 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345141 5054 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:38.361699 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345143 5054 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:38.361699 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345146 5054 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:38.361699 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345148 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:38.361699 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.345151 5054 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:38.361699 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.348690 5054 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 18:37:38.361699 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.348704 5054 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 18:37:38.361699 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.348720 5054 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 18:37:38.361699 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.348724 5054 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 18:37:38.361699 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348779 5054 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:38.361699 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348784 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:38.361699 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348787 5054 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:38.361699 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348790 5054 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:38.361699 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348793 5054 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:38.361699 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348796 5054 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:38.361699 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348798 5054 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:38.361699 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348801 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:38.361699 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348804 5054 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:38.361699 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348808 5054 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:38.361699 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348810 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:38.362205 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348813 5054 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:38.362205 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348816 5054 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:38.362205 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348819 5054 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:38.362205 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348822 5054 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:38.362205 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348825 5054 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:38.362205 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348828 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:38.362205 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348831 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:38.362205 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348833 5054 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:38.362205 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348836 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:38.362205 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348839 5054 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:38.362205 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348842 5054 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:38.362205 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348845 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:38.362205 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348848 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:38.362205 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348850 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:38.362205 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348853 5054 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:38.362205 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348856 5054 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:38.362205 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348859 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:38.362205 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348862 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:38.362205 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348865 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:38.362205 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348867 5054 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:38.362682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348870 5054 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:38.362682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348872 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:38.362682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348875 5054 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:38.362682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348877 5054 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:38.362682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348880 5054 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:38.362682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348883 5054 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:38.362682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348887 5054 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:38.362682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348890 5054 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:38.362682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348893 5054 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:38.362682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348897 5054 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:38.362682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348900 5054 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:38.362682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348904 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:38.362682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348906 5054 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:38.362682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348909 5054 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:38.362682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348912 5054 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:38.362682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348915 5054 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:38.362682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348917 5054 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:38.362682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348920 5054 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:38.362682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348922 5054 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:38.362682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348925 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:38.363195 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348928 5054 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:38.363195 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348930 5054 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:38.363195 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348933 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:38.363195 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348936 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:38.363195 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348940 5054 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:38.363195 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348944 5054 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:38.363195 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348947 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:38.363195 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348949 5054 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:38.363195 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348952 5054 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:38.363195 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348955 5054 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:38.363195 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348958 5054 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:38.363195 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348960 5054 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:38.363195 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348963 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:38.363195 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348965 5054 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:38.363195 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348968 5054 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:38.363195 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348970 5054 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:38.363195 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348973 5054 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:38.363195 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348976 5054 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:38.363195 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348978 5054 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:38.363670 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348981 5054 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:38.363670 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348983 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:38.363670 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348986 5054 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:38.363670 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348988 5054 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:38.363670 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348991 5054 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:38.363670 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348994 5054 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:38.363670 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.348996 5054 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:38.363670 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349011 5054 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:38.363670 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349016 5054 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:38.363670 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349019 5054 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:38.363670 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349021 5054 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:38.363670 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349024 5054 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:38.363670 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349027 5054 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:38.363670 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349029 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:38.363670 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349032 5054 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:38.363670 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349035 5054 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:38.363670 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349037 5054 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:38.363670 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349040 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:38.363670 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349042 5054 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:38.364140 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349045 5054 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:38.364140 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349048 5054 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:38.364140 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349050 5054 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:38.364140 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349053 5054 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:38.364140 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349056 5054 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:38.364140 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349058 5054 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:38.364140 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349061 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:38.364140 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349064 5054 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:38.364140 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349066 5054 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:38.364140 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349069 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:38.364140 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349071 5054 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:38.364140 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349074 5054 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:38.364140 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349077 5054 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:38.364140 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349079 5054 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:38.364140 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349082 5054 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:38.364140 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349085 5054 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:38.364140 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349087 5054 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:38.364606 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.349114 5054 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:37:38.364606 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349239 5054 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:38.364606 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349244 5054 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:38.364606 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349247 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:38.364606 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349251 5054 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:38.364606 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349255 5054 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:38.364606 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349258 5054 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:38.364606 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349261 5054 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:38.364606 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349263 5054 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:38.364606 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349266 5054 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:38.364606 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349269 5054 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:38.364606 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349272 5054 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:38.364606 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349275 5054 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:38.364606 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349277 5054 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:38.364606 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349280 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:38.364606 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349283 5054 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:38.365151 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349286 5054 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:38.365151 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349288 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:38.365151 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349291 5054 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:38.365151 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349294 5054 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:38.365151 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349297 5054 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:38.365151 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349299 5054 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:38.365151 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349302 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:38.365151 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349313 5054 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:38.365151 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349315 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:38.365151 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349318 5054 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:38.365151 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349320 5054 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:38.365151 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349323 5054 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:38.365151 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349326 5054 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:38.365151 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349329 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:38.365151 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349331 5054 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:38.365151 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349334 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:38.365151 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349339 5054 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:38.365151 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349342 5054 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:38.365151 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349345 5054 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:38.365762 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349347 5054 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:38.365762 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349351 5054 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:38.365762 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349355 5054 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:38.365762 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349358 5054 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:38.365762 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349361 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:38.365762 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349364 5054 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:38.365762 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349366 5054 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:38.365762 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349369 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:38.365762 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349372 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:38.365762 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349375 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:38.365762 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349378 5054 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:38.365762 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349380 5054 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:38.365762 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349383 5054 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:38.365762 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349385 5054 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:38.365762 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349388 5054 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:38.365762 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349390 5054 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:38.365762 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349393 5054 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:38.365762 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349396 5054 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:38.365762 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349401 5054 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:38.366246 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349405 5054 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:38.366246 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349410 5054 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:38.366246 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349414 5054 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:38.366246 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349418 5054 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:38.366246 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349422 5054 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:38.366246 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349426 5054 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:38.366246 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349430 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:38.366246 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349435 5054 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:38.366246 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349437 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:38.366246 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349440 5054 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:38.366246 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349444 5054 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:38.366246 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349447 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:38.366246 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349449 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:38.366246 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349452 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:38.366246 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349455 5054 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:38.366246 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349457 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:38.366246 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349460 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:38.366246 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349463 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:38.366246 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349465 5054 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:38.366246 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349468 5054 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:38.366725 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349470 5054 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:38.366725 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349473 5054 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:38.366725 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349475 5054 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:38.366725 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349478 5054 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:38.366725 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349481 5054 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:38.366725 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349483 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:38.366725 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349486 5054 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:38.366725 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349488 5054 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:38.366725 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349491 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:38.366725 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349494 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:38.366725 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349496 5054 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:38.366725 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349499 5054 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:38.366725 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349502 5054 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:38.366725 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349505 5054 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:38.366725 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349507 5054 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:38.366725 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349510 5054 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:38.366725 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349512 5054 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:38.366725 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349515 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:38.366725 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349518 5054 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:38.367224 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349520 5054 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:38.367224 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349523 5054 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:38.367224 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349526 5054 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:38.367224 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349529 5054 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:38.367224 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349532 5054 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:38.367224 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349535 5054 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:38.367224 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349537 5054 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:38.367224 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349540 5054 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:38.367224 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349543 5054 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:38.367224 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349546 5054 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:38.367224 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349548 5054 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:38.367224 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349551 5054 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:38.367224 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349553 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:38.367224 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349556 5054 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:38.367224 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.349581 5054 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:37:38.367224 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349613 5054 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:38.367630 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349617 5054 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:38.367630 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349621 5054 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:38.367630 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349624 5054 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:38.367630 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349627 5054 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:38.367630 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349629 5054 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:38.367630 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349632 5054 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:38.367630 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349635 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:38.367630 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349637 5054 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:38.367630 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349640 5054 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:38.367630 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349643 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:38.367630 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349646 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:38.367630 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349649 5054 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:38.367630 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349651 5054 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:38.367630 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349654 5054 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:38.367630 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349657 5054 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:38.367630 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349659 5054 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:38.367630 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349662 5054 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:38.367630 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349665 5054 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:38.367630 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349667 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:38.368280 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349670 5054 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:38.368280 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349674 5054 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:38.368280 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349676 5054 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:38.368280 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349679 5054 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:38.368280 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349681 5054 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:38.368280 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349684 5054 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:38.368280 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349686 5054 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:38.368280 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349689 5054 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:38.368280 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349692 5054 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:38.368280 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349694 5054 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:38.368280 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349697 5054 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:38.368280 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349699 5054 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:38.368280 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349702 5054 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:38.368280 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349704 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:38.368280 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349707 5054 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:38.368280 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349710 5054 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:38.368280 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349712 5054 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:38.368280 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349715 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:38.368280 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349719 5054 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:38.369092 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349723 5054 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:38.369092 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349726 5054 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:38.369092 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349729 5054 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:38.369092 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349732 5054 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:38.369092 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349734 5054 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:38.369092 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349737 5054 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:38.369092 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349740 5054 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:38.369092 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349743 5054 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:38.369092 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349745 5054 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:38.369092 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349748 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:38.369092 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349750 5054 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:38.369092 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349753 5054 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:38.369092 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349756 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:38.369092 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349758 5054 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:38.369092 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349763 5054 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:38.369092 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349766 5054 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:38.369092 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349768 5054 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:38.369092 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349771 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:38.369092 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349774 5054 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:38.369092 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349776 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:38.369900 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349779 5054 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:38.369900 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349782 5054 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:38.369900 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349784 5054 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:38.369900 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349787 5054 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:38.369900 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349790 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:38.369900 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349793 5054 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:38.369900 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349795 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:38.369900 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349798 5054 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:38.369900 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349801 5054 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:38.369900 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349803 5054 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:38.369900 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349806 5054 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:38.369900 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349810 5054 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:38.369900 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349814 5054 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:38.369900 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349818 5054 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:38.369900 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349821 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:38.369900 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349823 5054 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:38.369900 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349826 5054 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:38.369900 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349828 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:38.369900 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349831 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:38.370682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349834 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:38.370682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349837 5054 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:38.370682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349839 5054 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:38.370682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349842 5054 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:38.370682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349845 5054 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:38.370682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349847 5054 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:38.370682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349850 5054 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:38.370682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349854 5054 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:38.370682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349857 5054 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:38.370682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349860 5054 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:38.370682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349862 5054 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:38.370682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349865 5054 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:38.370682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349868 5054 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:38.370682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349870 5054 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:38.370682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349873 5054 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:38.370682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349876 5054 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:38.370682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349878 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:38.370682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349882 5054 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:38.370682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349885 5054 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:38.370682 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349887 5054 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:38.371216 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349890 5054 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:38.371216 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349892 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:38.371216 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349895 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:38.371216 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349897 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:38.371216 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349900 5054 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:38.371216 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349902 5054 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:38.371216 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349907 5054 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:38.371216 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:38.349910 5054 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:38.371216 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.349955 5054 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 18:37:38.371216 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.352063 5054 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Oct 30 18:37:38.371216 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.352139 5054 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Oct 30 18:37:38.371216 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.352412 5054 server.go:1014] "Starting client certificate rotation" Oct 30 18:37:38.371216 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.352538 5054 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 18:37:38.371216 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.352588 5054 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:31:56 +0000 UTC" deadline="2028-08-11 08:44:53.925659104 +0000 UTC" Oct 30 18:37:38.371216 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.352615 5054 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15614h7m15.573046497s" Oct 30 18:37:38.371216 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.353125 5054 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:37:38.371767 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.354740 5054 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:37:38.371767 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.356028 5054 log.go:25] "Validated CRI v1 runtime API" Oct 30 18:37:38.371767 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.357724 5054 log.go:25] "Validated CRI v1 image API" Oct 30 18:37:38.371767 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.358437 5054 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 18:37:38.371767 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.360376 5054 fs.go:135] Filesystem UUIDs: map[20945065-5c2f-495f-a16a-d494deccdd9a:/dev/nvme0n1p4 428c119f-d882-4cfe-bf7f-8579b734ec25:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Oct 30 18:37:38.371987 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.360392 5054 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/06aaf11fadf294f562fe5dfead8760ff636dcd908ca84d12a367ccabbb14f879/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/06aaf11fadf294f562fe5dfead8760ff636dcd908ca84d12a367ccabbb14f879/userdata/shm major:0 minor:218 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0e9cdc0cc1bebc95d5b3ec8adbeec0beee23bf505befc959d302298857d4a3fc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0e9cdc0cc1bebc95d5b3ec8adbeec0beee23bf505befc959d302298857d4a3fc/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0ebfc98f0252b4d4ac1ac940a893beb05eb0a976e9fecdec0573c37adc6f42d4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0ebfc98f0252b4d4ac1ac940a893beb05eb0a976e9fecdec0573c37adc6f42d4/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1c5f8f6c3b5058226e9e427143606a83a1fdea05c61212411e392526543e80c1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1c5f8f6c3b5058226e9e427143606a83a1fdea05c61212411e392526543e80c1/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/25485677ed922f926f62f71fd46ef2ee95a205b5e3ed9bd16dfe9ad3a7d07714/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/25485677ed922f926f62f71fd46ef2ee95a205b5e3ed9bd16dfe9ad3a7d07714/userdata/shm major:0 minor:344 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2f375283f7767fa35504e3aca70a46b4db0284adef37d35bff11ce72f10ac3d2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2f375283f7767fa35504e3aca70a46b4db0284adef37d35bff11ce72f10ac3d2/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3c8a0a6850f3ad80a626af13fa06f6b320b58224072762baa98b8e510c93131f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3c8a0a6850f3ad80a626af13fa06f6b320b58224072762baa98b8e510c93131f/userdata/shm major:0 minor:294 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4a00b3fe16bdd89c160b2ea6b60a3eb41037c475078a4aa24f4a18b7573c00e6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4a00b3fe16bdd89c160b2ea6b60a3eb41037c475078a4aa24f4a18b7573c00e6/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4dcdea706b660db0b90895641b499973489fd436605a40714e2efc108462bb7a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4dcdea706b660db0b90895641b499973489fd436605a40714e2efc108462bb7a/userdata/shm major:0 minor:204 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/52675bff9e6f9c76c989e6670a662df85a4741136b9da946053cc2102dc2f54b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/52675bff9e6f9c76c989e6670a662df85a4741136b9da946053cc2102dc2f54b/userdata/shm major:0 minor:342 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/53a0a315d1ae630f72c402a412e936660f22e80450846e2c9d0a0a05ecf09d35/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/53a0a315d1ae630f72c402a412e936660f22e80450846e2c9d0a0a05ecf09d35/userdata/shm major:0 minor:346 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/558cbff2e8507037d2164089aed5694c6267f1ddbf88050df2e780fdbc0bec89/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/558cbff2e8507037d2164089aed5694c6267f1ddbf88050df2e780fdbc0bec89/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6672e5ae55c20f12333e11f893df445070994a9ed7dd04e8037d93aa419b7b0f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6672e5ae55c20f12333e11f893df445070994a9ed7dd04e8037d93aa419b7b0f/userdata/shm major:0 minor:81 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6c336c6ab08a5c01702f4bee9b09fa45602c34a124233ec54a5394f16bcbd95d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6c336c6ab08a5c01702f4bee9b09fa45602c34a124233ec54a5394f16bcbd95d/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/784749f8c586b5ddfc3e84a7905c2212d47aa63e39a66a8277afbcd91d228b18/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/784749f8c586b5ddfc3e84a7905c2212d47aa63e39a66a8277afbcd91d228b18/userdata/shm major:0 minor:220 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7f59f42912f97c4914944345a46a250dbad8bda167a07f70972f2addbf6dc9b1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7f59f42912f97c4914944345a46a250dbad8bda167a07f70972f2addbf6dc9b1/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8e66dd96cdce3c9ee74b663d5a10f651c9b9e31f53a7783994ceb363539002ec/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8e66dd96cdce3c9ee74b663d5a10f651c9b9e31f53a7783994ceb363539002ec/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9e306e4abb478f7c63370fec934676c09baf889677a0d28fae31d6be10cb4d60/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9e306e4abb478f7c63370fec934676c09baf889677a0d28fae31d6be10cb4d60/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/acde37ead4bf7d34e9527554b79707fc35045e9df95cf19f8cd6e02118aa0949/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/acde37ead4bf7d34e9527554b79707fc35045e9df95cf19f8cd6e02118aa0949/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b4aab2d08faef690cce824193c9feb51340f503e938738e69988c5bd1f3a1895/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b4aab2d08faef690cce824193c9feb51340f503e938738e69988c5bd1f3a1895/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c286ec7589887ecf4b20167c2733b290cc9be04fb5b0479abab84f075c47828e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c286ec7589887ecf4b20167c2733b290cc9be04fb5b0479abab84f075c47828e/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c94e473384b14578f6885f47d13268485b29d20ed4b69f3881eb006a56e5984c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c94e473384b14578f6885f47d13268485b29d20ed4b69f3881eb006a56e5984c/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cc7cd7288990cb2519f5778f0589c8defb838504559335fca0d063c1d4160fb9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cc7cd7288990cb2519f5778f0589c8defb838504559335fca0d063c1d4160fb9/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d436d35f5c3b1730e19584bb15e329a39f0bb973334d2e7385433323968199f9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d436d35f5c3b1730e19584bb15e329a39f0bb973334d2e7385433323968199f9/userdata/shm major:0 minor:296 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e43a7a8c7a9b211e176baf848feebaf574c585874ab96cffee7861f821b25402/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e43a7a8c7a9b211e176baf848feebaf574c585874ab96cffee7861f821b25402/userdata/shm major:0 minor:212 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fb54c0a85ed5f3cc90ad6ef821488242fedb80ea7320f94810dbf3bf32ab10ea/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fb54c0a85ed5f3cc90ad6ef821488242fedb80ea7320f94810dbf3bf32ab10ea/userdata/shm major:0 minor:362 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/00d3be28-5482-48ea-a1be-629e8941236e/volumes/kubernetes.io~projected/kube-api-access-k4q88:{mountpoint:/var/lib/kubelet/pods/00d3be28-5482-48ea-a1be-629e8941236e/volumes/kubernetes.io~projected/kube-api-access-k4q88 major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/00d3be28-5482-48ea-a1be-629e8941236e/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/00d3be28-5482-48ea-a1be-629e8941236e/volumes/kubernetes.io~secret/serving-cert major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/03157316-915c-412d-9a93-5edcf607a91f/volumes/kubernetes.io~projected/kube-api-access-g4bc7:{mountpoint:/var/lib/kubelet/pods/03157316-915c-412d-9a93-5edcf607a91f/volumes/kubernetes.io~projected/kube-api-access-g4bc7 major:0 minor:335 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0c876fc3-6980-4a99-aed5-7439ca3d6b69/volumes/kubernetes.io~projected/kube-api-access-5gqkz:{mountpoint:/var/lib/kubelet/pods/0c876fc3-6980-4a99-aed5-7439ca3d6b69/volumes/kubernetes.io~projected/kube-api-access-5gqkz major:0 minor:232 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0c876fc3-6980-4a99-aed5-7439ca3d6b69/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/0c876fc3-6980-4a99-aed5-7439ca3d6b69/volumes/kubernetes.io~secret/serving-cert major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/10a7e4f4-2efc-4803-b3c2-da8749031462/volumes/kubernetes.io~projected/kube-api-access-nhpbc:{mountpoint:/var/lib/kubelet/pods/10a7e4f4-2efc-4803-b3c2-da8749031462/volumes/kubernetes.io~projected/kube-api-access-nhpbc major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/13bb697e-8bff-4a30-9e1d-b9f3baac416e/volumes/kubernetes.io~projected/kube-api-access-5glk7:{mountpoint:/var/lib/kubelet/pods/13bb697e-8bff-4a30-9e1d-b9f3baac416e/volumes/kubernetes.io~projected/kube-api-access-5glk7 major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/13bb697e-8bff-4a30-9e1d-b9f3baac416e/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/13bb697e-8bff-4a30-9e1d-b9f3baac416e/volumes/kubernetes.io~secret/serving-cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/14b832bf-1a7b-4539-811d-84a11d2b55b6/volumes/kubernetes.io~projected/kube-api-access-9lnmb:{mountpoint:/var/lib/kubelet/pods/14b832bf-1a7b-4539-811d-84a11d2b55b6/volumes/kubernetes.io~projected/kube-api-access-9lnmb major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7/volumes/kubernetes.io~projected/kube-api-access-wj9qm:{mountpoint:/var/lib/kubelet/pods/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7/volumes/kubernetes.io~projected/kube-api-access-wj9qm major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7/volumes/kubernetes.io~secret/serving-cert major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1bec85c3-72c2-46dc-9f68-8e80f1478422/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/1bec85c3-72c2-46dc-9f68-8e80f1478422/volumes/kubernetes.io~secret/agent-certs major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2297f719-7e19-4de8-b59c-f75885db9cc6/volumes/kubernetes.io~projected/kube-api-access-v6k6g:{mountpoint:/var/lib/kubelet/pods/2297f719-7e19-4de8-b59c-f75885db9cc6/volumes/kubernetes.io~projected/kube-api-access-v6k6g major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/25bce85b-a92c-4c39-a298-7b1881380069/volumes/kubernetes.io~projected/kube-api-access-fxznv:{mountpoint:/var/lib/kubelet/pods/25bce85b-a92c-4c39-a298-7b1881380069/volumes/kubernetes.io~projected/kube-api-access-fxznv major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a/volumes/kubernetes.io~projected/kube-api-access-nbxh5:{mountpoint:/var/lib/kubelet/pods/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a/volumes/kubernetes.io~projected/kube-api-access-nbxh5 major:0 minor:324 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/447f2b64-f13e-48cc-94b9-e80df4a9d4f3/volumes/kubernetes.io~projected/kube-api-access-q87cx:{mountpoint:/var/lib/kubelet/pods/447f2b64-f13e-48cc-94b9-e80df4a9d4f3/volumes/kubernetes.io~projected/kube-api-access-q87cx major:0 minor:228 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/542d052a-724f-4d98-9be7-7765a8c1d80a/volumes/kubernetes.io~projected/kube-api-access-fkd85:{mountpoint:/var/lib/kubelet/pods/542d052a-724f-4d98-9be7-7765a8c1d80a/volumes/kubernetes.io~projected/kube-api-access-fkd85 major:0 minor:231 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/542d052a-724f-4d98-9be7-7765a8c1d80a/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/542d052a-724f-4d98-9be7-7765a8c1d80a/volumes/kubernetes.io~secret/cert major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6050488d-f6ab-4bab-b2b2-90d30f64bab4/volumes/kubernetes.io~projected/kube-api-access-d8j7j:{mountpoint:/var/lib/kubelet/pods/6050488d-f6ab-4bab-b2b2-90d30f64bab4/volumes/kubernetes.io~projected/kube-api-access-d8j7j major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/66d0cf78-7aa4-40f2-ab29-dd33a5c73979/volumes/kubernetes.io~projected/kube-api-access-n7n2h:{mountpoint:/var/lib/kubelet/pods/66d0cf78-7aa4-40f2-ab29-dd33a5c73979/volumes/kubernetes.io~projected/kube-api-access-n7n2h major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/68788658-4afb-4757-acfa-7638cfb2488b/volumes/kubernetes.io~projected/kube-api-access-mjf45:{mountpoint:/var/lib/kubelet/pods/68788658-4afb-4757-acfa-7638cfb2488b/volumes/kubernetes.io~projected/kube-api-access-mjf45 major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6a29b358-3cb7-49e6-85a9-6581a2f52bd1/volumes/kubernetes.io~projected/kube-api-access-gbb9x:{mountpoint:/var/lib/kubelet/pods/6a29b358-3cb7-49e6-85a9-6581a2f52bd1/volumes/kubernetes.io~projected/kube-api-access-gbb9x major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/70781b52-75b2-450a-a058-b0d609659cda/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/70781b52-75b2-450a-a058-b0d609659cda/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/70781b52-75b2-450a-a058-b0d609659cda/volumes/kubernetes.io~projected/kube-api-access-tww25:{mountpoint:/var/lib/kubelet/pods/70781b52-75b2-450a-a058-b0d609659cda/volumes/kubernetes.io~projected/kube-api-access-tww25 major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/70781b52-75b2-450a-a058-b0d609659cda/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/70781b52-75b2-450a-a058-b0d609659cda/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/74153c90-cd72-43b8-85c5-aadb4d66a3e4/volumes/kubernetes.io~projected/kube-api-access-8w86l:{mountpoint:/var/lib/kubelet/pods/74153c90-cd72-43b8-85c5-aadb4d66a3e4/volumes/kubernetes.io~projected/kube-api-access-8w86l major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7fcdea8f-3b0f-42c5-bb3d-b2266b046491/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/7fcdea8f-3b0f-42c5-bb3d-b2266b046491/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:235 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7fcdea8f-3b0f-42c5-bb3d-b2266b046491/volumes/kubernetes.io~projected/kube-api-access-t24cp:{mountpoint:/var/lib/kubelet/pods/7fcdea8f-3b0f-42c5-bb3d-b2266b046491/volumes/kubernetes.io~projected/kube-api-access-t24cp major:0 minor:234 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7fcdea8f-3b0f-42c5-bb3d-b2266b046491/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/7fcdea8f-3b0f-42c5-bb3d-b2266b046491/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7fcdea8f-3b0f-42c5-bb3d-b2266b046491/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/7fcdea8f-3b0f-42c5-bb3d-b2266b046491/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7/volumes/kubernetes.io~projected/kube-api-access-flw7z:{mountpoint:/var/lib/kubelet/pods/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7/volumes/kubernetes.io~projected/kube-api-access-flw7z major:0 minor:233 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7/volumes/kubernetes.io~secret/default-certificate major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7/volumes/kubernetes.io~secret/stats-auth major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85fa03e6-458a-45e7-8c7b-a6820618a18a/volumes/kubernetes.io~projected/kube-api-access-2mpj6:{mountpoint:/var/lib/kubelet/pods/85fa03e6-458a-45e7-8c7b-a6820618a18a/volumes/kubernetes.io~projected/kube-api-access-2mpj6 major:0 minor:336 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8a3c8868-93e2-4f20-910e-d2cd73bc1d5f/volumes/kubernetes.io~projected/kube-api-access-v8pcp:{mountpoint:/var/lib/kubelet/pods/8a3c8868-93e2-4f20-910e-d2cd73bc1d5f/volumes/kubernetes.io~projected/kube-api-access-v8pcp major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/97294b18-463b-429d-8073-dc532ab0a046/volumes/kubernetes.io~projected/kube-api-access-tr22p:{mountpoint:/var/lib/kubelet/pods/97294b18-463b-429d-8073-dc532ab0a046/volumes/kubernetes.io~projected/kube-api-access-tr22p major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9b5d4a9f-0410-49ba-ab07-c0c69e3fe45f/volumes/kubernetes.io~projected/kube-api-access-twdrz:{mountpoint:/var/lib/kubelet/pods/9b5d4a9f-0410-49ba-ab07-c0c69e3fe45f/volumes/kubernetes.io~projected/kube-api-access-twdrz major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a74aa745-ed96-4880-ad11-a31391e40f57/volumes/kubernetes.io~projected/kube-api-access-k5chv:{mountpoint:/var/lib/kubelet/pods/a74aa745-ed96-4880-ad11-a31391e40f57/volumes/kubernetes.io~projected/kube-api-access-k5chv major:0 minor:361 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a74aa745-ed96-4880-ad11-a31391e40f57/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/a74aa745-ed96-4880-ad11-a31391e40f57/volumes/kubernetes.io~secret/signing-key major:0 minor:356 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b18682b8-f91a-4c27-87eb-82720ba4257f/volumes/kubernetes.io~projected/kube-api-access-5zf4n:{mountpoint:/var/lib/kubelet/pods/b18682b8-f91a-4c27-87eb-82720ba4257f/volumes/kubernetes.io~projected/kube-api-access-5zf4n major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b4a1d050-5a83-445c-a5a9-03ce4c64cc94/volumes/kubernetes.io~projected/kube-api-access-5z9kz:{mountpoint:/var/lib/kubelet/pods/b4a1d050-5a83-445c-a5a9-03ce4c64cc94/volumes/kubernetes.io~projected/kube-api-access-5z9kz major:0 minor:341 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b7ad340e-8e80-4f55-a29b-e8d501946cb5/volumes/kubernetes.io~projected/kube-api-access-ftcrx:{mountpoint:/var/lib/kubelet/pods/b7ad340e-8e80-4f55-a29b-e8d501946cb5/volumes/kubernetes.io~projected/kube-api-access-ftcrx major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d8443eba-410b-4aea-94b7-f5fdabbb06e3/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/d8443eba-410b-4aea-94b7-f5fdabbb06e3/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d8443eba-410b-4aea-94b7-f5fdabbb06e3/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/d8443eba-410b-4aea-94b7-f5fdabbb06e3/volumes/kubernetes.io~empty-dir/tmp major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d8443eba-410b-4aea-94b7-f5fdabbb06e3/volumes/kubernetes.io~projected/kube-api-access-9q64j:{mountpoint:/var/lib/kubelet/pods/d8443eba-410b-4aea-94b7-f5fdabbb06e3/volumes/kubernetes.io~projected/kube-api-access-9q64j major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d8e44bd9-13d6-4e54-9d9b-bfad33e1393c/volumes/kubernetes.io~projected/kube-api-access-l5tc8:{mountpoint:/var/lib/kubelet/pods/d8e44bd9-13d6-4e54-9d9b-bfad33e1393c/volumes/kubernetes.io~projected/kube-api-access-l5tc8 major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d9b465b9-97b6-47e2-8732-ed29b8f24ed0/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/d9b465b9-97b6-47e2-8732-ed29b8f24ed0/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d9b465b9-97b6-47e2-8732-ed29b8f24ed0/volumes/kubernetes.io~projected/kube-api-access-dw7kf:{mountpoint:/var/lib/kubelet/pods/d9b465b9-97b6-47e2-8732-ed29b8f24ed0/volumes/kubernetes.io~projected/kube-api-access-dw7kf major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d9b465b9-97b6-47e2-8732-ed29b8f24ed0/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/d9b465b9-97b6-47e2-8732-ed29b8f24ed0/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d9b465b9-97b6-47e2-8732-ed29b8f24ed0/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/d9b465b9-97b6-47e2-8732-ed29b8f24ed0/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f1cae505-4cef-46e5-85cc-cf8b410e654a/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/f1cae505-4cef-46e5-85cc-cf8b410e654a/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:288 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/ae1bd7e5ee76a5d23fbd040948275e49ed4836426290b17dfde686f65d7d2342/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/cbf79d2179f72205d92c07364e9a29604579e7efe986598a02500469cad87084/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/1c17cae8731916901baad72ed8ddb62e331dcf0fed3e509a1c3d6ff115209942/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/1d584eb677828bfe793541803f4de6d695aa4088b1bdc96f166901f5b55debee/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-111:{mountpoint:/var/lib/containers/storage/overlay/49e47334ffbe6eb55c4b07f4579e74e519b65f208c6bb347ccaa8a3116ed652e/merged major:0 minor:111 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/484cb3e7336f641796a1ecb977da65d9b48d584b3203d03235ab5a4f8b8f7530/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/0a8306a577ff88863636b3f587b143df6b2d01a1f25cd44d9b6034a71ec32c4b/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/f76d597d5e5dddd9f0878bc34c1a62697c949b50d5eabd6829ce5fa3907eb0c0/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-129:{mountpoint:/var/lib/containers/storage/overlay/458ffa890d5a72ccd9c04c31aa52e26c55c681d46f577dabd547263bf4741999/merged major:0 minor:129 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/c5fbf4161476877f8790c102266d0f8343e7b26c4f9c4b226b8fd672c729b2fe/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-138:{mountpoint:/var/lib/containers/storage/overlay/ff84e8833f37066d8133765124fd02251d9d76baf7a13e8d9d6ac2faafdbc7d4/merged major:0 minor:138 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/fd4523bbc0634eae8e38350298bbd91ed1c1f8bfd1a11ac0c77e0d618083222b/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/0ff62abec4c0bc666425be596c6751cb5a4e552b8de61455164449b8020ffe84/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/a5cd0ee6dbb109ca92ad7c2f8ca5c0e55d23fb9373534091a26c94b6181cd6f5/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/e6f2fb3a59015ff32108c3879a73cde294bbba8f225211c0de19f6ca8ff9d55e/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/fe371439f194df33f4c08ca36ed7d88d242a4ee1b7210550940e64c0a9b8344a/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/d5b48eea37d2eb598ec8ded4365d5d1adf296c6edc50c2c9b4a8c74250099263/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/75f7a1037beeb839d8484cf9a2951ad23b3394fb1301b4833d4de4dfda4a4269/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/d469ab25c73509263746a49978d63fa887d4dbabaa9acc03bdaacc9593ada6aa/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/a0894b053a17b1859589cf9a6851957a223089efd73f7d625d15e4e498e77a3d/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/920b50ed6c5addcd9eb874def9bc1c258cd08d663fd63eb9e13fd90da48d4116/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/903a0c9ad615d16d6387ac3dfa1a8f42522aff8d511ce981f2069a6d74a81cd0/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/a15568893306e721399cc7d9ebfb443eec6d1bed63e6245092ec8f20d9ed4403/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/9c3e5efd8c86c6980653896117e019e5251e94ced9ebdeace7da324e104f0688/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/c0541a16ed6b7dcc9d1d85bd9f962896688f11a9132d468eae10b56e27c124dc/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/dde33659296d69e04ea6d948d28322bc6859291463b908ff8cab32ca593b3fc7/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/39f5267b1594af71aac2d612cf1738dd189e52c7b6befdba88f2c9bac9c71159/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/f59204458b4daaf1f5f179f6ee448d37b4cf05036f0a5adbcdc180f33b7fc60e/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/0085385fd722b621c003c6874aacc0c20e79dab180d84319a6063081d2dd4639/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-263:{mountpoint:/var/lib/containers/storage/overlay/a673f880a2e9548668aa83551177fa05b47ba2653e26f790698ccdc031cd7347/merged major:0 minor:263 fsType:overlay blockSize:0} overlay_0-265:{mountpoint:/var/lib/containers/storage/overlay/0b3325acc34e6123f7520ad53df8857b38028cd722f5694d39b71173e44e8a36/merged major:0 minor:265 fsType:overlay blockSize:0} overlay_0-274:{mountpoint:/var/lib/containers/storage/overlay/01b69b9f677af913a68690771e9c8f984f457e9703c4a7624019e6e072f8bea7/merged major:0 minor:274 fsType:overlay blockSize:0} overlay_0-276:{mountpoint:/var/lib/containers/storage/overlay/8a3e8ab8a7efc813060af946815454542bac558e2b89a7dd71a302ae8be8aade/merged major:0 minor:276 fsType:overlay blockSize:0} overlay_0-278:{mountpoint:/var/lib/containers/storage/overlay/338eeba5cf7b0fca2f2bcdaf0fc3f154b4200b841305a5ae98512952a7d94231/merged major:0 minor:278 fsType:overlay blockSize:0} overlay_0-286:{mountpoint:/var/lib/containers/storage/overlay/3e172c3d0319d73eb16d2502d04e84dd62ba92ac9b47ab149772ce2ec9f9cc11/merged major:0 minor:286 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/377d321e4bbc73f971078c108fc7f5b20e879e5adde4e469bd393bd7b3ce551a/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/46922360865b2c76eedcdc1b64f6ad6d7bd1144910e1fcec0926a20fecc1a66f/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/7d65f2647500a8388c4f498f2ebccbd36f792f22906e40a55b6ad201ba7a3065/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/b8343263bf6a2183dcb3c88ac4edd9279d8a667fe052d82adc1a231b8a7b3588/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-306:{mountpoint:/var/lib/containers/storage/overlay/6f82829d515bd5f516a0f5090f648f0d0141f962ebeec0a26e2ca9d9dbb17f28/merged major:0 minor:306 fsType:overlay blockSize:0} overlay_0-308:{mountpoint:/var/lib/containers/storage/overlay/d446f8738b95634e0a75ddbb3ad584d91a8b5e4deecde97a1db06f7d342a23a2/merged major:0 minor:308 fsType:overlay blockSize:0} overlay_0-322:{mountpoint:/var/lib/containers/storage/overlay/5a04e71cf486111b0014b5d786c7c632851bde412b9e6ff28d3b691a04afb301/merged major:0 minor:322 fsType:overlay blockSize:0} overlay_0-329:{mountpoint:/var/lib/containers/storage/overlay/ee48ac9919535335385f269d3dc3f33ba3344c0f1734443c62cc678fd40347d6/merged major:0 minor:329 fsType:overlay blockSize:0} overlay_0-348:{mountpoint:/var/lib/containers/storage/overlay/c1463e956f211ec3aa21e2ab5c8e5e34b77946d805e05a27a758aa5eec69f529/merged major:0 minor:348 fsType:overlay blockSize:0} overlay_0-350:{mountpoint:/var/lib/containers/storage/overlay/7795201f8eab5c12e4c2939b0a07a28d63f296ce413c79335d74e6368e441748/merged major:0 minor:350 fsType:overlay blockSize:0} overlay_0-352:{mountpoint:/var/lib/containers/storage/overlay/b41e879411b691142f37bdb34de72ade245d69b9b4f78aef205513e8e3fa59e5/merged major:0 minor:352 fsType:overlay blockSize:0} overlay_0-354:{mountpoint:/var/lib/containers/storage/overlay/5e6e6b53b6489fe1fd33442604924b359d2aa1e7fb9270937655ae74ed5c0c38/merged major:0 minor:354 fsType:overlay blockSize:0} overlay_0-364:{mountpoint:/var/lib/containers/storage/overlay/9513d6543bc8900e3a83d5f978e584a810ff50689959875b3521f531e88e5499/merged major:0 minor:364 fsType:overlay blockSize:0} overlay_0-366:{mountpoint:/var/lib/containers/storage/overlay/012be4f6a5a544930eb0a3724c9719cc9d501d81f8b766f877212317d7ecac1a/merged major:0 minor:366 fsType:overlay blockSize:0} overlay_0-368:{mountpoint:/var/lib/containers/storage/overlay/9f960c59794eac921a4052624681f148093b7125912f3ae4dde955d292482af3/merged major:0 minor:368 fsType:overlay blockSize:0} overlay_0-374:{mountpoint:/var/lib/containers/storage/overlay/3548376a4e9871c0e637d81edf6e3d87d7c6ee77a31c6a71aab2c7b4a2b31c89/merged major:0 minor:374 fsType:overlay blockSize:0} overlay_0-376:{mountpoint:/var/lib/containers/storage/overlay/2b2e9b332c596ec0008b322fe58d27ba89bc1ce3440b907a6fd7b5794cb257eb/merged major:0 minor:376 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/d1a5ed31f8495bae501c47d6ef7b2140aec7c44cf8779668474211cca255d402/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/b3271294d49d860951f0385da9932e00d8b6bd1db01b77a1435a7412f371bdd8/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/03c221392fa121107f5e940105b335110620ff351d928f0d241ff98a2795effe/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/8ed85949de2bb711b5b5a5fce03f5410e9781a86142513a95948bccb8963efe8/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/e35e73231478d6a497604a4127a324d792cbac5469db1f162e919ca20c24573e/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/c9f55b2792bb6cf03e26c1a3ba961dd6a21cb21c8569c28840bc30816869386a/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/61d88768ac0cc3ad2ea279754ab02542624c83b951bec8cfdff2716f31461f8c/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/3479c907e34c4105662c44eeb1945cafb91570703a25aac94074ddaff9d15ebb/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/c55200f011faa07acdcfad17dcffc42979b048ea6dcd59301f2683dacc637c5a/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/7c6cc285f140f0249dba95568c47910e222db61d443885917a7efcd97f01ca5a/merged major:0 minor:96 fsType:overlay blockSize:0}] Oct 30 18:37:38.372366 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.369820 5054 manager.go:217] Machine: {Timestamp:2026-10-30 18:37:38.36881836 +0000 UTC m=+0.097197069 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099700 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ba733113a161fbb88a5b528f0ba92 SystemUUID:ec2ba733-113a-161f-bb88-a5b528f0ba92 BootID:a546173e-5f71-4725-bb31-5f4119027579 Filesystems:[{Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/06aaf11fadf294f562fe5dfead8760ff636dcd908ca84d12a367ccabbb14f879/userdata/shm DeviceMajor:0 DeviceMinor:218 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6672e5ae55c20f12333e11f893df445070994a9ed7dd04e8037d93aa419b7b0f/userdata/shm DeviceMajor:0 DeviceMinor:81 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b4a1d050-5a83-445c-a5a9-03ce4c64cc94/volumes/kubernetes.io~projected/kube-api-access-5z9kz DeviceMajor:0 DeviceMinor:341 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/53a0a315d1ae630f72c402a412e936660f22e80450846e2c9d0a0a05ecf09d35/userdata/shm DeviceMajor:0 DeviceMinor:346 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7/volumes/kubernetes.io~projected/kube-api-access-flw7z DeviceMajor:0 DeviceMinor:233 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/70781b52-75b2-450a-a058-b0d609659cda/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/13bb697e-8bff-4a30-9e1d-b9f3baac416e/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:45 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-276 DeviceMajor:0 DeviceMinor:276 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-364 DeviceMajor:0 DeviceMinor:364 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e43a7a8c7a9b211e176baf848feebaf574c585874ab96cffee7861f821b25402/userdata/shm DeviceMajor:0 DeviceMinor:212 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9e306e4abb478f7c63370fec934676c09baf889677a0d28fae31d6be10cb4d60/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d9b465b9-97b6-47e2-8732-ed29b8f24ed0/volumes/kubernetes.io~projected/kube-api-access-dw7kf DeviceMajor:0 DeviceMinor:225 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/85fa03e6-458a-45e7-8c7b-a6820618a18a/volumes/kubernetes.io~projected/kube-api-access-2mpj6 DeviceMajor:0 DeviceMinor:336 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/52675bff9e6f9c76c989e6670a662df85a4741136b9da946053cc2102dc2f54b/userdata/shm DeviceMajor:0 DeviceMinor:342 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7fcdea8f-3b0f-42c5-bb3d-b2266b046491/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:235 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4a00b3fe16bdd89c160b2ea6b60a3eb41037c475078a4aa24f4a18b7573c00e6/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-138 DeviceMajor:0 DeviceMinor:138 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7fcdea8f-3b0f-42c5-bb3d-b2266b046491/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:208 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:215 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0c876fc3-6980-4a99-aed5-7439ca3d6b69/volumes/kubernetes.io~projected/kube-api-access-5gqkz DeviceMajor:0 DeviceMinor:232 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7fcdea8f-3b0f-42c5-bb3d-b2266b046491/volumes/kubernetes.io~projected/kube-api-access-t24cp DeviceMajor:0 DeviceMinor:234 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1c5f8f6c3b5058226e9e427143606a83a1fdea05c61212411e392526543e80c1/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-111 DeviceMajor:0 DeviceMinor:111 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d9b465b9-97b6-47e2-8732-ed29b8f24ed0/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:209 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cc7cd7288990cb2519f5778f0589c8defb838504559335fca0d063c1d4160fb9/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-286 DeviceMajor:0 DeviceMinor:286 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/97294b18-463b-429d-8073-dc532ab0a046/volumes/kubernetes.io~projected/kube-api-access-tr22p DeviceMajor:0 DeviceMinor:293 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-306 DeviceMajor:0 DeviceMinor:306 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-322 DeviceMajor:0 DeviceMinor:322 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d8443eba-410b-4aea-94b7-f5fdabbb06e3/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:55 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6050488d-f6ab-4bab-b2b2-90d30f64bab4/volumes/kubernetes.io~projected/kube-api-access-d8j7j DeviceMajor:0 DeviceMinor:203 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/7fcdea8f-3b0f-42c5-bb3d-b2266b046491/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:211 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2297f719-7e19-4de8-b59c-f75885db9cc6/volumes/kubernetes.io~projected/kube-api-access-v6k6g DeviceMajor:0 DeviceMinor:230 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/03157316-915c-412d-9a93-5edcf607a91f/volumes/kubernetes.io~projected/kube-api-access-g4bc7 DeviceMajor:0 DeviceMinor:335 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/74153c90-cd72-43b8-85c5-aadb4d66a3e4/volumes/kubernetes.io~projected/kube-api-access-8w86l DeviceMajor:0 DeviceMinor:223 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/68788658-4afb-4757-acfa-7638cfb2488b/volumes/kubernetes.io~projected/kube-api-access-mjf45 DeviceMajor:0 DeviceMinor:226 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fb54c0a85ed5f3cc90ad6ef821488242fedb80ea7320f94810dbf3bf32ab10ea/userdata/shm DeviceMajor:0 DeviceMinor:362 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d8443eba-410b-4aea-94b7-f5fdabbb06e3/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:56 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b7ad340e-8e80-4f55-a29b-e8d501946cb5/volumes/kubernetes.io~projected/kube-api-access-ftcrx DeviceMajor:0 DeviceMinor:61 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-278 DeviceMajor:0 DeviceMinor:278 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-368 DeviceMajor:0 DeviceMinor:368 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/10a7e4f4-2efc-4803-b3c2-da8749031462/volumes/kubernetes.io~projected/kube-api-access-nhpbc DeviceMajor:0 DeviceMinor:66 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-129 DeviceMajor:0 DeviceMinor:129 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-354 DeviceMajor:0 DeviceMinor:354 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-376 DeviceMajor:0 DeviceMinor:376 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:210 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b18682b8-f91a-4c27-87eb-82720ba4257f/volumes/kubernetes.io~projected/kube-api-access-5zf4n DeviceMajor:0 DeviceMinor:222 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7/volumes/kubernetes.io~projected/kube-api-access-wj9qm DeviceMajor:0 DeviceMinor:224 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/447f2b64-f13e-48cc-94b9-e80df4a9d4f3/volumes/kubernetes.io~projected/kube-api-access-q87cx DeviceMajor:0 DeviceMinor:228 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c286ec7589887ecf4b20167c2733b290cc9be04fb5b0479abab84f075c47828e/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a74aa745-ed96-4880-ad11-a31391e40f57/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:356 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/70781b52-75b2-450a-a058-b0d609659cda/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:58 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6c336c6ab08a5c01702f4bee9b09fa45602c34a124233ec54a5394f16bcbd95d/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-348 DeviceMajor:0 DeviceMinor:348 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a74aa745-ed96-4880-ad11-a31391e40f57/volumes/kubernetes.io~projected/kube-api-access-k5chv DeviceMajor:0 DeviceMinor:361 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/558cbff2e8507037d2164089aed5694c6267f1ddbf88050df2e780fdbc0bec89/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/542d052a-724f-4d98-9be7-7765a8c1d80a/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:207 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/00d3be28-5482-48ea-a1be-629e8941236e/volumes/kubernetes.io~projected/kube-api-access-k4q88 DeviceMajor:0 DeviceMinor:229 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-366 DeviceMajor:0 DeviceMinor:366 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d8e44bd9-13d6-4e54-9d9b-bfad33e1393c/volumes/kubernetes.io~projected/kube-api-access-l5tc8 DeviceMajor:0 DeviceMinor:65 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-374 DeviceMajor:0 DeviceMinor:374 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d9b465b9-97b6-47e2-8732-ed29b8f24ed0/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:227 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-274 DeviceMajor:0 DeviceMinor:274 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d436d35f5c3b1730e19584bb15e329a39f0bb973334d2e7385433323968199f9/userdata/shm DeviceMajor:0 DeviceMinor:296 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/1bec85c3-72c2-46dc-9f68-8e80f1478422/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:57 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0ebfc98f0252b4d4ac1ac940a893beb05eb0a976e9fecdec0573c37adc6f42d4/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/13bb697e-8bff-4a30-9e1d-b9f3baac416e/volumes/kubernetes.io~projected/kube-api-access-5glk7 DeviceMajor:0 DeviceMinor:201 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/f1cae505-4cef-46e5-85cc-cf8b410e654a/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:288 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6a29b358-3cb7-49e6-85a9-6581a2f52bd1/volumes/kubernetes.io~projected/kube-api-access-gbb9x DeviceMajor:0 DeviceMinor:67 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c94e473384b14578f6885f47d13268485b29d20ed4b69f3881eb006a56e5984c/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7f59f42912f97c4914944345a46a250dbad8bda167a07f70972f2addbf6dc9b1/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-308 DeviceMajor:0 DeviceMinor:308 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-350 DeviceMajor:0 DeviceMinor:350 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-265 DeviceMajor:0 DeviceMinor:265 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3c8a0a6850f3ad80a626af13fa06f6b320b58224072762baa98b8e510c93131f/userdata/shm DeviceMajor:0 DeviceMinor:294 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d8443eba-410b-4aea-94b7-f5fdabbb06e3/volumes/kubernetes.io~projected/kube-api-access-9q64j DeviceMajor:0 DeviceMinor:59 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/70781b52-75b2-450a-a058-b0d609659cda/volumes/kubernetes.io~projected/kube-api-access-tww25 DeviceMajor:0 DeviceMinor:64 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9b5d4a9f-0410-49ba-ab07-c0c69e3fe45f/volumes/kubernetes.io~projected/kube-api-access-twdrz DeviceMajor:0 DeviceMinor:202 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/25485677ed922f926f62f71fd46ef2ee95a205b5e3ed9bd16dfe9ad3a7d07714/userdata/shm DeviceMajor:0 DeviceMinor:344 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8a3c8868-93e2-4f20-910e-d2cd73bc1d5f/volumes/kubernetes.io~projected/kube-api-access-v8pcp DeviceMajor:0 DeviceMinor:60 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-352 DeviceMajor:0 DeviceMinor:352 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0e9cdc0cc1bebc95d5b3ec8adbeec0beee23bf505befc959d302298857d4a3fc/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/66d0cf78-7aa4-40f2-ab29-dd33a5c73979/volumes/kubernetes.io~projected/kube-api-access-n7n2h DeviceMajor:0 DeviceMinor:62 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b4aab2d08faef690cce824193c9feb51340f503e938738e69988c5bd1f3a1895/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/acde37ead4bf7d34e9527554b79707fc35045e9df95cf19f8cd6e02118aa0949/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/542d052a-724f-4d98-9be7-7765a8c1d80a/volumes/kubernetes.io~projected/kube-api-access-fkd85 DeviceMajor:0 DeviceMinor:231 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2f375283f7767fa35504e3aca70a46b4db0284adef37d35bff11ce72f10ac3d2/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0c876fc3-6980-4a99-aed5-7439ca3d6b69/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:213 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:214 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-263 DeviceMajor:0 DeviceMinor:263 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/25bce85b-a92c-4c39-a298-7b1881380069/volumes/kubernetes.io~projected/kube-api-access-fxznv DeviceMajor:0 DeviceMinor:63 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8e66dd96cdce3c9ee74b663d5a10f651c9b9e31f53a7783994ceb363539002ec/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4dcdea706b660db0b90895641b499973489fd436605a40714e2efc108462bb7a/userdata/shm DeviceMajor:0 DeviceMinor:204 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d9b465b9-97b6-47e2-8732-ed29b8f24ed0/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:206 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/784749f8c586b5ddfc3e84a7905c2212d47aa63e39a66a8277afbcd91d228b18/userdata/shm DeviceMajor:0 DeviceMinor:220 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a/volumes/kubernetes.io~projected/kube-api-access-nbxh5 DeviceMajor:0 DeviceMinor:324 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/00d3be28-5482-48ea-a1be-629e8941236e/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:205 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/14b832bf-1a7b-4539-811d-84a11d2b55b6/volumes/kubernetes.io~projected/kube-api-access-9lnmb DeviceMajor:0 DeviceMinor:197 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-329 DeviceMajor:0 DeviceMinor:329 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:06aaf11fadf294f MacAddress:5e:fb:b9:79:a3:5e Speed:10000 Mtu:8901} {Name:25485677ed922f9 MacAddress:7a:57:b6:2e:68:b4 Speed:10000 Mtu:8901} {Name:2f375283f7767fa MacAddress:1a:8b:77:12:e9:bd Speed:10000 Mtu:8901} {Name:3c8a0a6850f3ad8 MacAddress:3e:24:4b:e1:d9:a9 Speed:10000 Mtu:8901} {Name:4dcdea706b660db MacAddress:7a:20:78:8a:57:f7 Speed:10000 Mtu:8901} {Name:52675bff9e6f9c7 MacAddress:c2:61:8e:d8:0d:7e Speed:10000 Mtu:8901} {Name:53a0a315d1ae630 MacAddress:32:16:ca:7a:bc:75 Speed:10000 Mtu:8901} {Name:6c336c6ab08a5c0 MacAddress:6e:24:d0:e1:fd:98 Speed:10000 Mtu:8901} {Name:784749f8c586b5d MacAddress:de:70:23:61:23:74 Speed:10000 Mtu:8901} {Name:7f59f42912f97c4 MacAddress:5a:49:9a:44:a3:46 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:ce:7e:29:71 Speed:0 Mtu:9001} {Name:br-int MacAddress:12:22:2b:89:47:a0 Speed:0 Mtu:8901} {Name:c286ec7589887ec MacAddress:ca:8b:7c:33:d8:67 Speed:10000 Mtu:8901} {Name:c94e473384b1457 MacAddress:42:6c:fe:4d:d0:50 Speed:10000 Mtu:8901} {Name:cc7cd7288990cb2 MacAddress:6a:7d:b2:90:4d:6f Speed:10000 Mtu:8901} {Name:d436d35f5c3b173 MacAddress:3a:f0:d1:84:8b:51 Speed:10000 Mtu:8901} {Name:e43a7a8c7a9b211 MacAddress:52:0f:ac:1d:34:1b Speed:10000 Mtu:8901} {Name:ens5 MacAddress:06:ff:ce:7e:29:71 Speed:0 Mtu:9001} {Name:fb54c0a85ed5f3c MacAddress:9a:0c:f6:86:6b:7b Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:c2:42:ee:78:8b:24 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:66:39:97:1c:72:39 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 18:37:38.372366 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.370472 5054 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 18:37:38.372366 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.370532 5054 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 18:37:38.372366 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.370808 5054 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 18:37:38.372366 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.370828 5054 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-74.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 18:37:38.372366 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.370942 5054 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 18:37:38.372366 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.370949 5054 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 18:37:38.372366 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.370959 5054 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:37:38.372366 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.370978 5054 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:37:38.372366 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.370995 5054 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 18:37:38.372366 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.371148 5054 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 18:37:38.372366 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.371240 5054 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 18:37:38.372366 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.371272 5054 kubelet.go:493] "Attempting to sync node with API server" Oct 30 18:37:38.372366 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.371287 5054 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 18:37:38.372366 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.371297 5054 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 18:37:38.372366 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.371305 5054 kubelet.go:405] "Adding apiserver pod source" Oct 30 18:37:38.372366 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.371313 5054 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 18:37:38.372366 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.371975 5054 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 18:37:38.372366 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.371985 5054 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:37:38.373049 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.372419 5054 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 18:37:38.375360 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.375344 5054 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 18:37:38.375360 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.375358 5054 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:37:38.375464 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.375454 5054 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Oct 30 18:37:38.375630 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.375619 5054 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 18:37:38.375671 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.375658 5054 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 18:37:38.375758 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.375749 5054 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 18:37:38.375788 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.375762 5054 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 18:37:38.375788 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.375769 5054 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 18:37:38.375788 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.375775 5054 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 18:37:38.375788 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.375781 5054 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 18:37:38.375788 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.375787 5054 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 18:37:38.375928 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.375792 5054 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 18:37:38.375928 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.375801 5054 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 18:37:38.375928 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.375811 5054 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 18:37:38.375928 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.375817 5054 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 18:37:38.375928 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.375825 5054 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 18:37:38.375928 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.375835 5054 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 18:37:38.375928 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.375860 5054 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 18:37:38.375928 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.375867 5054 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 18:37:38.379215 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.379201 5054 server.go:1263] "Started kubelet" Oct 30 18:37:38.379319 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.379289 5054 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 18:37:38.379791 ip-10-0-132-74 systemd[1]: Started Kubernetes Kubelet. Oct 30 18:37:38.382491 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.381809 5054 server.go:317] "Adding debug handlers to kubelet server" Oct 30 18:37:38.382491 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.380247 5054 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 18:37:38.382491 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.382146 5054 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 18:37:38.384051 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.382772 5054 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 18:37:38.387197 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.387176 5054 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 18:37:38.387281 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.387202 5054 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 18:37:38.387281 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.387268 5054 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:31:58 +0000 UTC" deadline="2028-05-18 20:39:45.062404162 +0000 UTC" Oct 30 18:37:38.387394 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.387286 5054 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13586h2m6.67512132s" Oct 30 18:37:38.387394 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.387305 5054 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 18:37:38.387394 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.387315 5054 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 18:37:38.387394 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:38.387316 5054 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-74.us-west-2.compute.internal\" not found" Oct 30 18:37:38.387874 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.387720 5054 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.388462 5054 factory.go:55] Registering systemd factory Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.388487 5054 factory.go:223] Registration of the systemd container factory successfully Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.388750 5054 factory.go:153] Registering CRI-O factory Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.388766 5054 factory.go:223] Registration of the crio container factory successfully Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.388822 5054 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.388844 5054 factory.go:103] Registering Raw factory Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.388861 5054 manager.go:1199] Started watching for new ooms in manager Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.389205 5054 manager.go:319] Starting recovery of all containers Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391622 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d9b465b9-97b6-47e2-8732-ed29b8f24ed0" volumeName="kubernetes.io/configmap/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-trusted-ca" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391652 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="03157316-915c-412d-9a93-5edcf607a91f" volumeName="kubernetes.io/configmap/03157316-915c-412d-9a93-5edcf607a91f-service-ca-bundle" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391667 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="10a7e4f4-2efc-4803-b3c2-da8749031462" volumeName="kubernetes.io/configmap/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-daemon-config" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391678 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7fcdea8f-3b0f-42c5-bb3d-b2266b046491" volumeName="kubernetes.io/configmap/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-trusted-ca" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391692 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00d3be28-5482-48ea-a1be-629e8941236e" volumeName="kubernetes.io/empty-dir/00d3be28-5482-48ea-a1be-629e8941236e-tmp" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391706 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0c876fc3-6980-4a99-aed5-7439ca3d6b69" volumeName="kubernetes.io/configmap/0c876fc3-6980-4a99-aed5-7439ca3d6b69-config" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391719 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="447f2b64-f13e-48cc-94b9-e80df4a9d4f3" volumeName="kubernetes.io/projected/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-kube-api-access-q87cx" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391743 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="542d052a-724f-4d98-9be7-7765a8c1d80a" volumeName="kubernetes.io/projected/542d052a-724f-4d98-9be7-7765a8c1d80a-kube-api-access-fkd85" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391758 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b7ad340e-8e80-4f55-a29b-e8d501946cb5" volumeName="kubernetes.io/projected/b7ad340e-8e80-4f55-a29b-e8d501946cb5-kube-api-access-ftcrx" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391770 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1704ffe1-d69c-4dd2-b7f6-5880bf1551f7" volumeName="kubernetes.io/secret/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7-serving-cert" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391782 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3ccc9d41-f2e7-4b0d-a064-cb3e5549049a" volumeName="kubernetes.io/projected/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-kube-api-access-nbxh5" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391793 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70781b52-75b2-450a-a058-b0d609659cda" volumeName="kubernetes.io/configmap/70781b52-75b2-450a-a058-b0d609659cda-env-overrides" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391805 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7" volumeName="kubernetes.io/projected/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-kube-api-access-flw7z" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391817 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b18682b8-f91a-4c27-87eb-82720ba4257f" volumeName="kubernetes.io/configmap/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-config" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391832 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d9b465b9-97b6-47e2-8732-ed29b8f24ed0" volumeName="kubernetes.io/configmap/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-certificates" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391858 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13bb697e-8bff-4a30-9e1d-b9f3baac416e" volumeName="kubernetes.io/secret/13bb697e-8bff-4a30-9e1d-b9f3baac416e-serving-cert" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391903 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="25bce85b-a92c-4c39-a298-7b1881380069" volumeName="kubernetes.io/projected/25bce85b-a92c-4c39-a298-7b1881380069-kube-api-access-fxznv" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391918 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70781b52-75b2-450a-a058-b0d609659cda" volumeName="kubernetes.io/configmap/70781b52-75b2-450a-a058-b0d609659cda-ovnkube-config" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391933 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70781b52-75b2-450a-a058-b0d609659cda" volumeName="kubernetes.io/secret/70781b52-75b2-450a-a058-b0d609659cda-ovn-node-metrics-cert" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391946 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7" volumeName="kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-stats-auth" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391959 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d9b465b9-97b6-47e2-8732-ed29b8f24ed0" volumeName="kubernetes.io/secret/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-installation-pull-secrets" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391974 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00d3be28-5482-48ea-a1be-629e8941236e" volumeName="kubernetes.io/configmap/00d3be28-5482-48ea-a1be-629e8941236e-trusted-ca-bundle" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.391989 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0c876fc3-6980-4a99-aed5-7439ca3d6b69" volumeName="kubernetes.io/configmap/0c876fc3-6980-4a99-aed5-7439ca3d6b69-trusted-ca" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392018 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7fcdea8f-3b0f-42c5-bb3d-b2266b046491" volumeName="kubernetes.io/secret/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-image-registry-private-configuration" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392030 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00d3be28-5482-48ea-a1be-629e8941236e" volumeName="kubernetes.io/empty-dir/00d3be28-5482-48ea-a1be-629e8941236e-snapshots" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392045 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3ccc9d41-f2e7-4b0d-a064-cb3e5549049a" volumeName="kubernetes.io/empty-dir/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-data-volume" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392058 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="66d0cf78-7aa4-40f2-ab29-dd33a5c73979" volumeName="kubernetes.io/configmap/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-cni-sysctl-allowlist" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392088 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7fcdea8f-3b0f-42c5-bb3d-b2266b046491" volumeName="kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-kube-api-access-t24cp" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392108 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="97294b18-463b-429d-8073-dc532ab0a046" volumeName="kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392121 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d8443eba-410b-4aea-94b7-f5fdabbb06e3" volumeName="kubernetes.io/empty-dir/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-tuned" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392134 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0c876fc3-6980-4a99-aed5-7439ca3d6b69" volumeName="kubernetes.io/projected/0c876fc3-6980-4a99-aed5-7439ca3d6b69-kube-api-access-5gqkz" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392164 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1bec85c3-72c2-46dc-9f68-8e80f1478422" volumeName="kubernetes.io/secret/1bec85c3-72c2-46dc-9f68-8e80f1478422-agent-certs" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392177 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="447f2b64-f13e-48cc-94b9-e80df4a9d4f3" volumeName="kubernetes.io/empty-dir/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-tmp-dir" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392190 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6a29b358-3cb7-49e6-85a9-6581a2f52bd1" volumeName="kubernetes.io/configmap/6a29b358-3cb7-49e6-85a9-6581a2f52bd1-serviceca" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392203 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7fcdea8f-3b0f-42c5-bb3d-b2266b046491" volumeName="kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-bound-sa-token" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392215 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a3c8868-93e2-4f20-910e-d2cd73bc1d5f" volumeName="kubernetes.io/configmap/8a3c8868-93e2-4f20-910e-d2cd73bc1d5f-iptables-alerter-script" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392225 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a74aa745-ed96-4880-ad11-a31391e40f57" volumeName="kubernetes.io/projected/a74aa745-ed96-4880-ad11-a31391e40f57-kube-api-access-k5chv" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392237 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d8443eba-410b-4aea-94b7-f5fdabbb06e3" volumeName="kubernetes.io/empty-dir/d8443eba-410b-4aea-94b7-f5fdabbb06e3-tmp" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392250 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00d3be28-5482-48ea-a1be-629e8941236e" volumeName="kubernetes.io/configmap/00d3be28-5482-48ea-a1be-629e8941236e-service-ca-bundle" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392260 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70781b52-75b2-450a-a058-b0d609659cda" volumeName="kubernetes.io/configmap/70781b52-75b2-450a-a058-b0d609659cda-ovnkube-script-lib" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392280 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="74153c90-cd72-43b8-85c5-aadb4d66a3e4" volumeName="kubernetes.io/projected/74153c90-cd72-43b8-85c5-aadb4d66a3e4-kube-api-access-8w86l" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392291 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85fa03e6-458a-45e7-8c7b-a6820618a18a" volumeName="kubernetes.io/projected/85fa03e6-458a-45e7-8c7b-a6820618a18a-kube-api-access-2mpj6" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392301 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d9b465b9-97b6-47e2-8732-ed29b8f24ed0" volumeName="kubernetes.io/empty-dir/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-ca-trust-extracted" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392312 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2297f719-7e19-4de8-b59c-f75885db9cc6" volumeName="kubernetes.io/projected/2297f719-7e19-4de8-b59c-f75885db9cc6-kube-api-access-v6k6g" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392323 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00d3be28-5482-48ea-a1be-629e8941236e" volumeName="kubernetes.io/projected/00d3be28-5482-48ea-a1be-629e8941236e-kube-api-access-k4q88" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392343 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0c876fc3-6980-4a99-aed5-7439ca3d6b69" volumeName="kubernetes.io/secret/0c876fc3-6980-4a99-aed5-7439ca3d6b69-serving-cert" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392355 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="542d052a-724f-4d98-9be7-7765a8c1d80a" volumeName="kubernetes.io/secret/542d052a-724f-4d98-9be7-7765a8c1d80a-cert" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392366 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6a29b358-3cb7-49e6-85a9-6581a2f52bd1" volumeName="kubernetes.io/projected/6a29b358-3cb7-49e6-85a9-6581a2f52bd1-kube-api-access-gbb9x" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392378 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="70781b52-75b2-450a-a058-b0d609659cda" volumeName="kubernetes.io/projected/70781b52-75b2-450a-a058-b0d609659cda-kube-api-access-tww25" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392388 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="74153c90-cd72-43b8-85c5-aadb4d66a3e4" volumeName="kubernetes.io/configmap/74153c90-cd72-43b8-85c5-aadb4d66a3e4-telemetry-config" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392399 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b4a1d050-5a83-445c-a5a9-03ce4c64cc94" volumeName="kubernetes.io/projected/b4a1d050-5a83-445c-a5a9-03ce4c64cc94-kube-api-access-5z9kz" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392410 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1704ffe1-d69c-4dd2-b7f6-5880bf1551f7" volumeName="kubernetes.io/configmap/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7-config" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392421 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="66d0cf78-7aa4-40f2-ab29-dd33a5c73979" volumeName="kubernetes.io/configmap/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-whereabouts-flatfile-configmap" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392432 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="66d0cf78-7aa4-40f2-ab29-dd33a5c73979" volumeName="kubernetes.io/configmap/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-cni-binary-copy" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392443 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d8443eba-410b-4aea-94b7-f5fdabbb06e3" volumeName="kubernetes.io/projected/d8443eba-410b-4aea-94b7-f5fdabbb06e3-kube-api-access-9q64j" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392454 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d8e44bd9-13d6-4e54-9d9b-bfad33e1393c" volumeName="kubernetes.io/projected/d8e44bd9-13d6-4e54-9d9b-bfad33e1393c-kube-api-access-l5tc8" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392487 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="10a7e4f4-2efc-4803-b3c2-da8749031462" volumeName="kubernetes.io/configmap/10a7e4f4-2efc-4803-b3c2-da8749031462-cni-binary-copy" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392499 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="10a7e4f4-2efc-4803-b3c2-da8749031462" volumeName="kubernetes.io/projected/10a7e4f4-2efc-4803-b3c2-da8749031462-kube-api-access-nhpbc" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392511 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="66d0cf78-7aa4-40f2-ab29-dd33a5c73979" volumeName="kubernetes.io/projected/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-kube-api-access-n7n2h" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392523 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7" volumeName="kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-default-certificate" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392536 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d8e44bd9-13d6-4e54-9d9b-bfad33e1393c" volumeName="kubernetes.io/empty-dir/d8e44bd9-13d6-4e54-9d9b-bfad33e1393c-tmp-dir" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392548 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d9b465b9-97b6-47e2-8732-ed29b8f24ed0" volumeName="kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-kube-api-access-dw7kf" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392560 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f1cae505-4cef-46e5-85cc-cf8b410e654a" volumeName="kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392576 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1bec85c3-72c2-46dc-9f68-8e80f1478422" volumeName="kubernetes.io/configmap/1bec85c3-72c2-46dc-9f68-8e80f1478422-konnectivity-ca" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392589 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7fcdea8f-3b0f-42c5-bb3d-b2266b046491" volumeName="kubernetes.io/secret/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-installation-pull-secrets" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392601 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1340fb2d-5f77-4316-a156-f82e0cb1124d" volumeName="kubernetes.io/configmap/1340fb2d-5f77-4316-a156-f82e0cb1124d-nginx-conf" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392613 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13bb697e-8bff-4a30-9e1d-b9f3baac416e" volumeName="kubernetes.io/configmap/13bb697e-8bff-4a30-9e1d-b9f3baac416e-config" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392624 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="447f2b64-f13e-48cc-94b9-e80df4a9d4f3" volumeName="kubernetes.io/configmap/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-config-volume" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392638 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7fcdea8f-3b0f-42c5-bb3d-b2266b046491" volumeName="kubernetes.io/configmap/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-certificates" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392648 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8a3c8868-93e2-4f20-910e-d2cd73bc1d5f" volumeName="kubernetes.io/projected/8a3c8868-93e2-4f20-910e-d2cd73bc1d5f-kube-api-access-v8pcp" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392660 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00d3be28-5482-48ea-a1be-629e8941236e" volumeName="kubernetes.io/secret/00d3be28-5482-48ea-a1be-629e8941236e-serving-cert" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392671 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="03157316-915c-412d-9a93-5edcf607a91f" volumeName="kubernetes.io/configmap/03157316-915c-412d-9a93-5edcf607a91f-trusted-ca-bundle" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392683 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="03157316-915c-412d-9a93-5edcf607a91f" volumeName="kubernetes.io/empty-dir/03157316-915c-412d-9a93-5edcf607a91f-archives-path" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392694 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a74aa745-ed96-4880-ad11-a31391e40f57" volumeName="kubernetes.io/secret/a74aa745-ed96-4880-ad11-a31391e40f57-signing-key" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392704 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d9b465b9-97b6-47e2-8732-ed29b8f24ed0" volumeName="kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-bound-sa-token" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392716 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d9b465b9-97b6-47e2-8732-ed29b8f24ed0" volumeName="kubernetes.io/secret/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-image-registry-private-configuration" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392728 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="03157316-915c-412d-9a93-5edcf607a91f" volumeName="kubernetes.io/projected/03157316-915c-412d-9a93-5edcf607a91f-kube-api-access-g4bc7" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392739 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="13bb697e-8bff-4a30-9e1d-b9f3baac416e" volumeName="kubernetes.io/projected/13bb697e-8bff-4a30-9e1d-b9f3baac416e-kube-api-access-5glk7" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392753 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="14b832bf-1a7b-4539-811d-84a11d2b55b6" volumeName="kubernetes.io/projected/14b832bf-1a7b-4539-811d-84a11d2b55b6-kube-api-access-9lnmb" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392765 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6050488d-f6ab-4bab-b2b2-90d30f64bab4" volumeName="kubernetes.io/projected/6050488d-f6ab-4bab-b2b2-90d30f64bab4-kube-api-access-d8j7j" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392776 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="68788658-4afb-4757-acfa-7638cfb2488b" volumeName="kubernetes.io/projected/68788658-4afb-4757-acfa-7638cfb2488b-kube-api-access-mjf45" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392787 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7fcdea8f-3b0f-42c5-bb3d-b2266b046491" volumeName="kubernetes.io/empty-dir/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-ca-trust-extracted" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392800 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b5d4a9f-0410-49ba-ab07-c0c69e3fe45f" volumeName="kubernetes.io/projected/9b5d4a9f-0410-49ba-ab07-c0c69e3fe45f-kube-api-access-twdrz" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392811 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a74aa745-ed96-4880-ad11-a31391e40f57" volumeName="kubernetes.io/configmap/a74aa745-ed96-4880-ad11-a31391e40f57-signing-cabundle" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392824 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1704ffe1-d69c-4dd2-b7f6-5880bf1551f7" volumeName="kubernetes.io/projected/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7-kube-api-access-wj9qm" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392836 5054 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b18682b8-f91a-4c27-87eb-82720ba4257f" volumeName="kubernetes.io/projected/b18682b8-f91a-4c27-87eb-82720ba4257f-kube-api-access-5zf4n" seLinuxMountContext="" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392847 5054 reconstruct.go:97] "Volume reconstruction finished" Oct 30 18:37:38.393195 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.392854 5054 reconciler.go:29] "Reconciler: start to sync state" Oct 30 18:37:38.412613 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.412588 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 18:37:38.412756 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.412738 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 18:37:38.413968 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.413934 5054 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 18:37:38.415567 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.415541 5054 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 18:37:38.415567 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.415562 5054 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 18:37:38.415707 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.415581 5054 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 18:37:38.415707 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:38.415622 5054 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Oct 30 18:37:38.429298 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.429257 5054 generic.go:356] "Generic (PLEG): container finished" podID="66d0cf78-7aa4-40f2-ab29-dd33a5c73979" containerID="b0286eb871d8e27520f053cfc4524dc67767b66b0848bb534a8a1be04d9d0edc" exitCode=0 Oct 30 18:37:38.429298 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.429282 5054 generic.go:356] "Generic (PLEG): container finished" podID="66d0cf78-7aa4-40f2-ab29-dd33a5c73979" containerID="abdb22ff8a3c4899daad63c816c67c33a147e7ec76a392d52f46859033d546de" exitCode=0 Oct 30 18:37:38.429298 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.429291 5054 generic.go:356] "Generic (PLEG): container finished" podID="66d0cf78-7aa4-40f2-ab29-dd33a5c73979" containerID="b99c31f6cba7810ab9c68a7e292526361b870a082a3b0256717db2149c7a32b5" exitCode=0 Oct 30 18:37:38.429298 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.429299 5054 generic.go:356] "Generic (PLEG): container finished" podID="66d0cf78-7aa4-40f2-ab29-dd33a5c73979" containerID="c308cdf08f3f821ca679a1ff83ca8e7fe22c93698f58df805d06708b9ea415cf" exitCode=0 Oct 30 18:37:38.429568 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.429306 5054 generic.go:356] "Generic (PLEG): container finished" podID="66d0cf78-7aa4-40f2-ab29-dd33a5c73979" containerID="16772c9016af1b2d147ba8ac46d0ef276b8d1dccca6abd60fdccd6546392300b" exitCode=0 Oct 30 18:37:38.429568 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.429312 5054 generic.go:356] "Generic (PLEG): container finished" podID="66d0cf78-7aa4-40f2-ab29-dd33a5c73979" containerID="15b9618f25cc8ee09487bc989b2e2c920d504c3673120dd66d63c767287e79a4" exitCode=0 Oct 30 18:37:38.435423 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.435401 5054 generic.go:356] "Generic (PLEG): container finished" podID="6a99a11e235547936507bfb99804f96f" containerID="d90db88c72aeee52c7b6465eff8b29333b67359fdf62d6a008174f4357dfb5bd" exitCode=0 Oct 30 18:37:38.449916 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.449858 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 18:37:38.449916 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.449893 5054 generic.go:356] "Generic (PLEG): container finished" podID="0c876fc3-6980-4a99-aed5-7439ca3d6b69" containerID="5b64ecda12ea0b0a79e9aae4b5a3057c653329e1c1e98c72fb2411e0caca677e" exitCode=255 Oct 30 18:37:38.486533 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.486509 5054 manager.go:324] Recovery completed Oct 30 18:37:38.487421 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:38.487398 5054 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-74.us-west-2.compute.internal\" not found" Oct 30 18:37:38.505659 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.505638 5054 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:37:38.506790 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.506768 5054 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:37:38.506863 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.506802 5054 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:37:38.506863 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.506812 5054 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasSufficientPID" Oct 30 18:37:38.509403 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.509378 5054 cpu_manager.go:226] "Starting" policy="none" Oct 30 18:37:38.509403 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.509397 5054 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 18:37:38.509403 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.509399 5054 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:37:38.509580 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.509414 5054 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 18:37:38.509580 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.509534 5054 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Oct 30 18:37:38.509580 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.509546 5054 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Oct 30 18:37:38.509580 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.509562 5054 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Oct 30 18:37:38.509580 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.509568 5054 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Oct 30 18:37:38.509580 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.509573 5054 policy_none.go:50] "Start" Oct 30 18:37:38.509580 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.509581 5054 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 18:37:38.509848 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.509591 5054 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 18:37:38.509848 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.509709 5054 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Oct 30 18:37:38.509848 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.509718 5054 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Oct 30 18:37:38.509848 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.509725 5054 policy_none.go:44] "Start" Oct 30 18:37:38.510283 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.510263 5054 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:37:38.510333 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.510294 5054 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:37:38.510333 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.510305 5054 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasSufficientPID" Oct 30 18:37:38.514072 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.514049 5054 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 18:37:38.515948 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:38.515928 5054 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Oct 30 18:37:38.516686 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.516675 5054 manager.go:341] "Starting Device Plugin manager" Oct 30 18:37:38.516733 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:38.516697 5054 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 18:37:38.516733 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.516706 5054 server.go:87] "Starting device plugin registration server" Oct 30 18:37:38.516956 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.516946 5054 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 18:37:38.516992 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.516959 5054 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 18:37:38.517063 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.517049 5054 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 18:37:38.517166 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.517137 5054 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 18:37:38.517230 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.517173 5054 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 18:37:38.517230 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.517180 5054 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 18:37:38.517321 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.517242 5054 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:37:38.517163919Z","UUID":"a1c268e4-f9d0-4d5a-81c0-a508000ebb73","Handler":null,"Name":"","Endpoint":""} Oct 30 18:37:38.518799 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:38.518780 5054 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 18:37:38.518956 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:38.518942 5054 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-74.us-west-2.compute.internal\" not found" Oct 30 18:37:38.519315 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.519300 5054 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 18:37:38.519467 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.519455 5054 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 18:37:38.588890 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.588870 5054 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Oct 30 18:37:38.606184 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.606164 5054 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:37:38.606459 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.606442 5054 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:37:38.606537 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.606500 5054 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:37:38.617797 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.617779 5054 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:37:38.618807 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.618791 5054 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:37:38.618901 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.618822 5054 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:37:38.618901 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.618862 5054 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeHasSufficientPID" Oct 30 18:37:38.618998 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.618920 5054 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:37:38.654873 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.654854 5054 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:37:38.717052 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.716967 5054 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal"] Oct 30 18:37:38.717586 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.717554 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" event={"ID":"6a99a11e235547936507bfb99804f96f","Type":"ContainerStarted","Data":"272e8b415a40600a7873b9e65a4bbd3ade2469e9b210b2ace6dd1701765841ee"} Oct 30 18:37:38.717639 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.717593 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" event={"ID":"6a99a11e235547936507bfb99804f96f","Type":"ContainerDied","Data":"d90db88c72aeee52c7b6465eff8b29333b67359fdf62d6a008174f4357dfb5bd"} Oct 30 18:37:38.717639 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.717606 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" event={"ID":"6a99a11e235547936507bfb99804f96f","Type":"ContainerStarted","Data":"0e9cdc0cc1bebc95d5b3ec8adbeec0beee23bf505befc959d302298857d4a3fc"} Oct 30 18:37:38.717705 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.717674 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal" event={"ID":"958b813bafdd4852b0fbfdc2a07cf4ae","Type":"ContainerStarted","Data":"86ae2d7ac5c0b4bab8e6719ca2a39775a183030b6b26100416d3fae09434cd54"} Oct 30 18:37:38.717705 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.717687 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal" event={"ID":"958b813bafdd4852b0fbfdc2a07cf4ae","Type":"ContainerStarted","Data":"558cbff2e8507037d2164089aed5694c6267f1ddbf88050df2e780fdbc0bec89"} Oct 30 18:37:38.717705 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.717696 5054 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:37:38.717799 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.717789 5054 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:37:38.743016 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.742985 5054 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:37:38.743078 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.743068 5054 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:37:38.743627 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.743613 5054 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeNotReady" Oct 30 18:37:38.743704 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.743642 5054 setters.go:546] "Node became not ready" node="ip-10-0-132-74.us-west-2.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-10-30T18:37:38Z","lastTransitionTime":"2026-10-30T18:37:38Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"} Oct 30 18:37:38.789966 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.789935 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/958b813bafdd4852b0fbfdc2a07cf4ae-config\") pod \"kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal\" (UID: \"958b813bafdd4852b0fbfdc2a07cf4ae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:37:38.790084 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.789972 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a99a11e235547936507bfb99804f96f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal\" (UID: \"6a99a11e235547936507bfb99804f96f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:37:38.790084 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.789991 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a99a11e235547936507bfb99804f96f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal\" (UID: \"6a99a11e235547936507bfb99804f96f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:37:38.791160 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.791136 5054 nodeinfomanager.go:471] existing CSINode "ip-10-0-132-74.us-west-2.compute.internal" is owned by different node (oldNodeID="6b9e1f11-db7f-40ff-9bd7-79c5759e3917", newNodeID=""), cleaning up... Oct 30 18:37:38.815239 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.815218 5054 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:37:38.815315 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:38.815263 5054 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:37:38.816263 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.816245 5054 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:37:38.816341 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:38.816285 5054 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:37:38.890729 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.890707 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/958b813bafdd4852b0fbfdc2a07cf4ae-config\") pod \"kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal\" (UID: \"958b813bafdd4852b0fbfdc2a07cf4ae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:37:38.890809 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.890736 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a99a11e235547936507bfb99804f96f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal\" (UID: \"6a99a11e235547936507bfb99804f96f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:37:38.890809 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.890752 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a99a11e235547936507bfb99804f96f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal\" (UID: \"6a99a11e235547936507bfb99804f96f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:37:38.890809 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.890800 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a99a11e235547936507bfb99804f96f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal\" (UID: \"6a99a11e235547936507bfb99804f96f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:37:38.890907 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.890815 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a99a11e235547936507bfb99804f96f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal\" (UID: \"6a99a11e235547936507bfb99804f96f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:37:38.890907 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:38.890803 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/958b813bafdd4852b0fbfdc2a07cf4ae-config\") pod \"kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal\" (UID: \"958b813bafdd4852b0fbfdc2a07cf4ae\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:37:39.019850 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.019779 5054 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-74.us-west-2.compute.internal" event="NodeReady" Oct 30 18:37:39.019962 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.019867 5054 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Oct 30 18:37:39.372239 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.372215 5054 apiserver.go:52] "Watching apiserver" Oct 30 18:37:39.475820 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.475796 5054 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:37:39.475966 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.475948 5054 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:37:39.522023 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.521983 5054 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 18:37:39.524459 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.524425 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-kbclw","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq","openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6","openshift-cluster-node-tuning-operator/tuned-fbfdv","openshift-image-registry/node-ca-mxjc4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv","openshift-insights/insights-operator-7c4986d449-q9rgp","kube-system/kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal","openshift-network-operator/iptables-alerter-p88j5","openshift-dns/dns-default-ltkzx","openshift-network-diagnostics/network-check-source-6f78d59f99-d9fjm","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b","openshift-ingress-canary/ingress-canary-rbmpn","kube-system/kas-connection-checker-685895c7c9-mzqcg","kube-system/konnectivity-agent-w2wtr","kube-system/global-pull-secret-syncer-pxndl","openshift-network-console/networking-console-plugin-86c478bb88-dl96p","openshift-ingress/router-default-6c975679c5-x5lg8","openshift-kube-storage-version-migrator/migrator-66df5d496b-tsbpt","openshift-multus/network-metrics-daemon-phngs","openshift-network-diagnostics/network-check-target-62vfn","openshift-ovn-kubernetes/ovnkube-node-4hsc7","kube-system/kas-connection-checker-685895c7c9-k4dpj","openshift-kube-storage-version-migrator/migrator-66df5d496b-7kvml","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal","openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt","openshift-image-registry/image-registry-5df4b54dc9-8pdtm","openshift-multus/multus-928bp","openshift-insights/insights-runtime-extractor-bscj4","openshift-service-ca/service-ca-5fffc97bdf-rdt4m","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-nprgd","openshift-console-operator/console-operator-7cd56b8dbc-tp7sc","openshift-image-registry/image-registry-649485556b-7x5gc","openshift-insights/periodic-gathering-lnkcs-v7pdj","openshift-multus/multus-additional-cni-plugins-7dkzl","openshift-dns/node-resolver-k7r7n"] Oct 30 18:37:39.526873 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.526855 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:39.527235 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.527221 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:39.527601 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.527576 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:39.529607 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.529591 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:39.530036 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.530021 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" Oct 30 18:37:39.533544 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.533526 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:39.537310 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.537293 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:39.537880 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.537865 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:39.539453 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.539436 5054 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:37:39.539824 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.539809 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:39.541246 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.541230 5054 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:37:39.567961 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:39.567938 5054 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:37:39.568060 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:39.567971 5054 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal" Oct 30 18:37:39.580730 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.580708 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.580730 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.580707 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.580830 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.580708 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.580830 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.580750 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.581021 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.580988 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.581605 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.581589 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Oct 30 18:37:39.583780 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.583750 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Oct 30 18:37:39.583780 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.583770 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Oct 30 18:37:39.583925 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.583772 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Oct 30 18:37:39.583925 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.583828 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Oct 30 18:37:39.583925 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.583852 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Oct 30 18:37:39.583925 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.583897 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-hvfl9\"" Oct 30 18:37:39.584107 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.584041 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Oct 30 18:37:39.584817 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.584796 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.584817 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.584813 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-smlbs\"" Oct 30 18:37:39.584947 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.584805 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-5hnsq\"" Oct 30 18:37:39.585149 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.585127 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Oct 30 18:37:39.585446 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.585424 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Oct 30 18:37:39.585704 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.585676 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.585894 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.585874 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.585953 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.585906 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.586041 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.585967 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.588111 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.588090 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.588437 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.588420 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Oct 30 18:37:39.588619 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.588604 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-swfq5\"" Oct 30 18:37:39.588752 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.588738 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.588893 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.588879 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-tbwc5\"" Oct 30 18:37:39.588977 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.588956 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Oct 30 18:37:39.589044 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.589020 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\"" Oct 30 18:37:39.589044 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.589031 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.589148 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.589094 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.589148 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.589104 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.589148 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.589035 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.589148 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.589142 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-db58r\"" Oct 30 18:37:39.589320 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.589305 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Oct 30 18:37:39.589361 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.589316 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Oct 30 18:37:39.589361 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.589326 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.591626 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.591609 5054 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 18:37:39.593628 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.593611 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4q88\" (UniqueName: \"kubernetes.io/projected/00d3be28-5482-48ea-a1be-629e8941236e-kube-api-access-k4q88\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:39.593676 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.593637 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-data-volume\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:39.593676 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.593652 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7-config\") pod \"service-ca-operator-544d749c8c-b2rv6\" (UID: \"1704ffe1-d69c-4dd2-b7f6-5880bf1551f7\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6" Oct 30 18:37:39.593676 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.593670 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-image-registry-private-configuration\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:39.593773 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.593694 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw7kf\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-kube-api-access-dw7kf\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:39.593773 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.593760 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-data-volume\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:39.593867 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.593834 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-cnibin\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:37:39.593903 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.593884 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7-config\") pod \"service-ca-operator-544d749c8c-b2rv6\" (UID: \"1704ffe1-d69c-4dd2-b7f6-5880bf1551f7\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6" Oct 30 18:37:39.593903 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.593888 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-crio-socket\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:39.593962 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.593914 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-cni-netd\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.593962 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.593946 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-tuned\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.594038 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.593980 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-cni-dir\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.594085 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594035 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-tuned\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.594085 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594054 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-hostroot\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.594146 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594085 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zf4n\" (UniqueName: \"kubernetes.io/projected/b18682b8-f91a-4c27-87eb-82720ba4257f-kube-api-access-5zf4n\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:39.594146 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594111 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:39.594213 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594198 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:39.594242 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594214 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/00d3be28-5482-48ea-a1be-629e8941236e-serving-cert\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:39.594274 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594247 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-system-cni-dir\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.594274 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594261 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-sys-fs\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:37:39.594340 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594322 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1bec85c3-72c2-46dc-9f68-8e80f1478422-agent-certs\") pod \"konnectivity-agent-w2wtr\" (UID: \"1bec85c3-72c2-46dc-9f68-8e80f1478422\") " pod="kube-system/konnectivity-agent-w2wtr" Oct 30 18:37:39.594370 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594352 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-run-ovn\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.594424 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594413 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-kubernetes\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.594493 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594449 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mpj6\" (UniqueName: \"kubernetes.io/projected/85fa03e6-458a-45e7-8c7b-a6820618a18a-kube-api-access-2mpj6\") pod \"migrator-66df5d496b-tsbpt\" (UID: \"85fa03e6-458a-45e7-8c7b-a6820618a18a\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-tsbpt" Oct 30 18:37:39.594493 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594473 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:39.594614 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594601 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-stats-auth\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:39.594644 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594624 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/542d052a-724f-4d98-9be7-7765a8c1d80a-cert\") pod \"ingress-canary-rbmpn\" (UID: \"542d052a-724f-4d98-9be7-7765a8c1d80a\") " pod="openshift-ingress-canary/ingress-canary-rbmpn" Oct 30 18:37:39.594644 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594639 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:39.594703 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594654 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6a29b358-3cb7-49e6-85a9-6581a2f52bd1-serviceca\") pod \"node-ca-mxjc4\" (UID: \"6a29b358-3cb7-49e6-85a9-6581a2f52bd1\") " pod="openshift-image-registry/node-ca-mxjc4" Oct 30 18:37:39.594703 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594668 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lnmb\" (UniqueName: \"kubernetes.io/projected/14b832bf-1a7b-4539-811d-84a11d2b55b6-kube-api-access-9lnmb\") pod \"kas-connection-checker-685895c7c9-k4dpj\" (UID: \"14b832bf-1a7b-4539-811d-84a11d2b55b6\") " pod="kube-system/kas-connection-checker-685895c7c9-k4dpj" Oct 30 18:37:39.594703 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594683 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-default-certificate\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:39.594829 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594812 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-stats-auth\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:39.594947 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594927 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr22p\" (UniqueName: \"kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p\") pod \"network-check-target-62vfn\" (UID: \"97294b18-463b-429d-8073-dc532ab0a046\") " pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:39.595020 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594965 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tww25\" (UniqueName: \"kubernetes.io/projected/70781b52-75b2-450a-a058-b0d609659cda-kube-api-access-tww25\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.595020 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594982 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-default-certificate\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:39.595020 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.594992 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d3be28-5482-48ea-a1be-629e8941236e-service-ca-bundle\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:39.595154 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595101 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-trusted-ca\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:39.595154 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595128 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-installation-pull-secrets\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:39.595288 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595155 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-run-systemd\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.595288 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595182 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-var-lib-openvswitch\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.595288 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595233 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5glk7\" (UniqueName: \"kubernetes.io/projected/13bb697e-8bff-4a30-9e1d-b9f3baac416e-kube-api-access-5glk7\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-gktbq\" (UID: \"13bb697e-8bff-4a30-9e1d-b9f3baac416e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq" Oct 30 18:37:39.595288 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595273 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f1cae505-4cef-46e5-85cc-cf8b410e654a-dbus\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:39.595427 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595295 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:39.595427 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595318 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:39.595427 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595338 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-config-volume\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:39.595427 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595385 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d8e44bd9-13d6-4e54-9d9b-bfad33e1393c-hosts-file\") pod \"node-resolver-k7r7n\" (UID: \"d8e44bd9-13d6-4e54-9d9b-bfad33e1393c\") " pod="openshift-dns/node-resolver-k7r7n" Oct 30 18:37:39.595577 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595428 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-node-log\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.595577 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595470 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-run-ovn-kubernetes\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.595577 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595503 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-certificates\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:39.595577 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595527 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-etc-openvswitch\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.595710 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595580 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:37:39.595710 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595608 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/74153c90-cd72-43b8-85c5-aadb4d66a3e4-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:39.595809 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595720 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twdrz\" (UniqueName: \"kubernetes.io/projected/9b5d4a9f-0410-49ba-ab07-c0c69e3fe45f-kube-api-access-twdrz\") pod \"kas-connection-checker-685895c7c9-kbclw\" (UID: \"9b5d4a9f-0410-49ba-ab07-c0c69e3fe45f\") " pod="kube-system/kas-connection-checker-685895c7c9-kbclw" Oct 30 18:37:39.595809 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595750 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0c876fc3-6980-4a99-aed5-7439ca3d6b69-config\") pod \"console-operator-7cd56b8dbc-tp7sc\" (UID: \"0c876fc3-6980-4a99-aed5-7439ca3d6b69\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:39.595809 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595775 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/74153c90-cd72-43b8-85c5-aadb4d66a3e4-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:39.595809 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595776 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:39.595970 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595821 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/70781b52-75b2-450a-a058-b0d609659cda-ovnkube-config\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.595970 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595845 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/70781b52-75b2-450a-a058-b0d609659cda-ovnkube-script-lib\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.595970 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595891 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/13bb697e-8bff-4a30-9e1d-b9f3baac416e-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-gktbq\" (UID: \"13bb697e-8bff-4a30-9e1d-b9f3baac416e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq" Oct 30 18:37:39.595970 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595926 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:37:39.595970 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595946 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0c876fc3-6980-4a99-aed5-7439ca3d6b69-config\") pod \"console-operator-7cd56b8dbc-tp7sc\" (UID: \"0c876fc3-6980-4a99-aed5-7439ca3d6b69\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:39.595970 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595954 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8a3c8868-93e2-4f20-910e-d2cd73bc1d5f-iptables-alerter-script\") pod \"iptables-alerter-p88j5\" (UID: \"8a3c8868-93e2-4f20-910e-d2cd73bc1d5f\") " pod="openshift-network-operator/iptables-alerter-p88j5" Oct 30 18:37:39.596256 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595977 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/10a7e4f4-2efc-4803-b3c2-da8749031462-cni-binary-copy\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.596256 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.595994 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-var-lib-cni-bin\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.596256 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596036 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-bound-sa-token\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:39.596256 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596057 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/13bb697e-8bff-4a30-9e1d-b9f3baac416e-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-gktbq\" (UID: \"13bb697e-8bff-4a30-9e1d-b9f3baac416e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq" Oct 30 18:37:39.596256 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596102 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-system-cni-dir\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:37:39.596256 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596151 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-image-registry-private-configuration\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:39.596256 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596214 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-ca-trust-extracted\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:39.596539 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596259 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/03157316-915c-412d-9a93-5edcf607a91f-archives-path\") pod \"periodic-gathering-lnkcs-v7pdj\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:37:39.596539 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596312 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/03157316-915c-412d-9a93-5edcf607a91f-archives-path\") pod \"periodic-gathering-lnkcs-v7pdj\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:37:39.596539 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596317 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhpbc\" (UniqueName: \"kubernetes.io/projected/10a7e4f4-2efc-4803-b3c2-da8749031462-kube-api-access-nhpbc\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.596539 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596331 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-ca-trust-extracted\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:39.596539 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596342 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1340fb2d-5f77-4316-a156-f82e0cb1124d-nginx-conf\") pod \"networking-console-plugin-86c478bb88-dl96p\" (UID: \"1340fb2d-5f77-4316-a156-f82e0cb1124d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" Oct 30 18:37:39.596539 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596359 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-cni-binary-copy\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:37:39.596539 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596374 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d3be28-5482-48ea-a1be-629e8941236e-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:39.596539 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596393 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-modprobe-d\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.596539 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596426 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-sysctl-d\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.596539 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596482 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-run\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.596539 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596532 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w86l\" (UniqueName: \"kubernetes.io/projected/74153c90-cd72-43b8-85c5-aadb4d66a3e4-kube-api-access-8w86l\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:39.596982 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596562 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:37:39.596982 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596587 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-trusted-ca\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:39.596982 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596610 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7n2h\" (UniqueName: \"kubernetes.io/projected/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-kube-api-access-n7n2h\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:37:39.596982 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596637 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03157316-915c-412d-9a93-5edcf607a91f-service-ca-bundle\") pod \"periodic-gathering-lnkcs-v7pdj\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:37:39.596982 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596663 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5chv\" (UniqueName: \"kubernetes.io/projected/a74aa745-ed96-4880-ad11-a31391e40f57-kube-api-access-k5chv\") pod \"service-ca-5fffc97bdf-rdt4m\" (UID: \"a74aa745-ed96-4880-ad11-a31391e40f57\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rdt4m" Oct 30 18:37:39.596982 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596689 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-var-lib-cni-multus\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.596982 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596716 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:37:39.596982 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596742 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-run-multus-certs\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.596982 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596766 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-slash\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.596982 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596792 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-run-openvswitch\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.596982 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596837 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-socket-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:37:39.596982 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596885 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-bound-sa-token\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:39.596982 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596926 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a3c8868-93e2-4f20-910e-d2cd73bc1d5f-host-slash\") pod \"iptables-alerter-p88j5\" (UID: \"8a3c8868-93e2-4f20-910e-d2cd73bc1d5f\") " pod="openshift-network-operator/iptables-alerter-p88j5" Oct 30 18:37:39.596982 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596951 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-kubelet\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.596982 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.596977 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-device-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:37:39.597483 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597044 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gqkz\" (UniqueName: \"kubernetes.io/projected/0c876fc3-6980-4a99-aed5-7439ca3d6b69-kube-api-access-5gqkz\") pod \"console-operator-7cd56b8dbc-tp7sc\" (UID: \"0c876fc3-6980-4a99-aed5-7439ca3d6b69\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:39.597483 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597064 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00d3be28-5482-48ea-a1be-629e8941236e-tmp\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:39.597483 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597084 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs\") pod \"network-metrics-daemon-phngs\" (UID: \"25bce85b-a92c-4c39-a298-7b1881380069\") " pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:39.597483 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597110 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-log-socket\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.597483 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597136 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1bec85c3-72c2-46dc-9f68-8e80f1478422-konnectivity-ca\") pod \"konnectivity-agent-w2wtr\" (UID: \"1bec85c3-72c2-46dc-9f68-8e80f1478422\") " pod="kube-system/konnectivity-agent-w2wtr" Oct 30 18:37:39.597483 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597163 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxznv\" (UniqueName: \"kubernetes.io/projected/25bce85b-a92c-4c39-a298-7b1881380069-kube-api-access-fxznv\") pod \"network-metrics-daemon-phngs\" (UID: \"25bce85b-a92c-4c39-a298-7b1881380069\") " pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:39.597483 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597163 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/00d3be28-5482-48ea-a1be-629e8941236e-tmp\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:39.597483 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597275 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkd85\" (UniqueName: \"kubernetes.io/projected/542d052a-724f-4d98-9be7-7765a8c1d80a-kube-api-access-fkd85\") pod \"ingress-canary-rbmpn\" (UID: \"542d052a-724f-4d98-9be7-7765a8c1d80a\") " pod="openshift-ingress-canary/ingress-canary-rbmpn" Oct 30 18:37:39.597483 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597323 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbb9x\" (UniqueName: \"kubernetes.io/projected/6a29b358-3cb7-49e6-85a9-6581a2f52bd1-kube-api-access-gbb9x\") pod \"node-ca-mxjc4\" (UID: \"6a29b358-3cb7-49e6-85a9-6581a2f52bd1\") " pod="openshift-image-registry/node-ca-mxjc4" Oct 30 18:37:39.597483 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597397 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-run-k8s-cni-cncf-io\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.597483 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597482 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj9qm\" (UniqueName: \"kubernetes.io/projected/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7-kube-api-access-wj9qm\") pod \"service-ca-operator-544d749c8c-b2rv6\" (UID: \"1704ffe1-d69c-4dd2-b7f6-5880bf1551f7\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6" Oct 30 18:37:39.597804 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597531 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:39.597804 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597561 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/00d3be28-5482-48ea-a1be-629e8941236e-snapshots\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:39.597804 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597580 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-certificates\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:39.597804 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597598 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t24cp\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-kube-api-access-t24cp\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:39.597804 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597632 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03157316-915c-412d-9a93-5edcf607a91f-trusted-ca-bundle\") pod \"periodic-gathering-lnkcs-v7pdj\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:37:39.597804 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597659 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a74aa745-ed96-4880-ad11-a31391e40f57-signing-cabundle\") pod \"service-ca-5fffc97bdf-rdt4m\" (UID: \"a74aa745-ed96-4880-ad11-a31391e40f57\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rdt4m" Oct 30 18:37:39.597804 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597664 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/00d3be28-5482-48ea-a1be-629e8941236e-snapshots\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:39.597804 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597684 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8pcp\" (UniqueName: \"kubernetes.io/projected/8a3c8868-93e2-4f20-910e-d2cd73bc1d5f-kube-api-access-v8pcp\") pod \"iptables-alerter-p88j5\" (UID: \"8a3c8868-93e2-4f20-910e-d2cd73bc1d5f\") " pod="openshift-network-operator/iptables-alerter-p88j5" Oct 30 18:37:39.597804 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597711 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-cni-bin\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.597804 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597736 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5tc8\" (UniqueName: \"kubernetes.io/projected/d8e44bd9-13d6-4e54-9d9b-bfad33e1393c-kube-api-access-l5tc8\") pod \"node-resolver-k7r7n\" (UID: \"d8e44bd9-13d6-4e54-9d9b-bfad33e1393c\") " pod="openshift-dns/node-resolver-k7r7n" Oct 30 18:37:39.598272 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597803 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:39.598272 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597830 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-systemd-units\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.598272 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597854 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-sysconfig\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.598272 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597892 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-cnibin\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.598272 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597928 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftcrx\" (UniqueName: \"kubernetes.io/projected/b7ad340e-8e80-4f55-a29b-e8d501946cb5-kube-api-access-ftcrx\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:37:39.598272 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597955 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-metrics-certs\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:39.598272 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.597983 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flw7z\" (UniqueName: \"kubernetes.io/projected/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-kube-api-access-flw7z\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:39.598272 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598031 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbxh5\" (UniqueName: \"kubernetes.io/projected/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-kube-api-access-nbxh5\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:39.598272 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598065 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/70781b52-75b2-450a-a058-b0d609659cda-env-overrides\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.598272 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598091 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-socket-dir-parent\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.598272 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598116 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-run-netns\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.598272 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598141 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-registration-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:37:39.598272 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598177 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0c876fc3-6980-4a99-aed5-7439ca3d6b69-serving-cert\") pod \"console-operator-7cd56b8dbc-tp7sc\" (UID: \"0c876fc3-6980-4a99-aed5-7439ca3d6b69\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:39.598272 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598210 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-ca-trust-extracted\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:39.598272 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598240 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-dl96p\" (UID: \"1340fb2d-5f77-4316-a156-f82e0cb1124d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" Oct 30 18:37:39.598272 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598268 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6k6g\" (UniqueName: \"kubernetes.io/projected/2297f719-7e19-4de8-b59c-f75885db9cc6-kube-api-access-v6k6g\") pod \"volume-data-source-validator-58c9f5f46d-nprgd\" (UID: \"2297f719-7e19-4de8-b59c-f75885db9cc6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-nprgd" Oct 30 18:37:39.598888 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598285 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7-serving-cert\") pod \"service-ca-operator-544d749c8c-b2rv6\" (UID: \"1704ffe1-d69c-4dd2-b7f6-5880bf1551f7\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6" Oct 30 18:37:39.598888 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598294 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-ca-trust-extracted\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:39.598888 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598325 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-systemd\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.598888 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598359 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-var-lib-kubelet\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.598888 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598391 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-daemon-config\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.598888 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598428 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7-serving-cert\") pod \"service-ca-operator-544d749c8c-b2rv6\" (UID: \"1704ffe1-d69c-4dd2-b7f6-5880bf1551f7\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6" Oct 30 18:37:39.598888 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598546 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f1cae505-4cef-46e5-85cc-cf8b410e654a-kubelet-config\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:39.598888 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598579 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a74aa745-ed96-4880-ad11-a31391e40f57-signing-key\") pod \"service-ca-5fffc97bdf-rdt4m\" (UID: \"a74aa745-ed96-4880-ad11-a31391e40f57\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rdt4m" Oct 30 18:37:39.598888 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598598 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a29b358-3cb7-49e6-85a9-6581a2f52bd1-host\") pod \"node-ca-mxjc4\" (UID: \"6a29b358-3cb7-49e6-85a9-6581a2f52bd1\") " pod="openshift-image-registry/node-ca-mxjc4" Oct 30 18:37:39.598888 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598616 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8j7j\" (UniqueName: \"kubernetes.io/projected/6050488d-f6ab-4bab-b2b2-90d30f64bab4-kube-api-access-d8j7j\") pod \"kas-connection-checker-685895c7c9-mzqcg\" (UID: \"6050488d-f6ab-4bab-b2b2-90d30f64bab4\") " pod="kube-system/kas-connection-checker-685895c7c9-mzqcg" Oct 30 18:37:39.598888 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598637 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-tmp-dir\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:39.598888 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598648 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0c876fc3-6980-4a99-aed5-7439ca3d6b69-serving-cert\") pod \"console-operator-7cd56b8dbc-tp7sc\" (UID: \"0c876fc3-6980-4a99-aed5-7439ca3d6b69\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:39.598888 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598695 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4bc7\" (UniqueName: \"kubernetes.io/projected/03157316-915c-412d-9a93-5edcf607a91f-kube-api-access-g4bc7\") pod \"periodic-gathering-lnkcs-v7pdj\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:37:39.598888 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598703 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-tmp-dir\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:39.598888 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598730 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-run-netns\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.598888 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598756 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-host\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.598888 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598783 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-etc-selinux\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:37:39.598888 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598813 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-installation-pull-secrets\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:39.598888 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598847 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:39.599563 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598919 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8e44bd9-13d6-4e54-9d9b-bfad33e1393c-tmp-dir\") pod \"node-resolver-k7r7n\" (UID: \"d8e44bd9-13d6-4e54-9d9b-bfad33e1393c\") " pod="openshift-dns/node-resolver-k7r7n" Oct 30 18:37:39.599563 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.598967 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/70781b52-75b2-450a-a058-b0d609659cda-ovn-node-metrics-cert\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.599563 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.599023 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-sysctl-conf\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.599563 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.599045 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8e44bd9-13d6-4e54-9d9b-bfad33e1393c-tmp-dir\") pod \"node-resolver-k7r7n\" (UID: \"d8e44bd9-13d6-4e54-9d9b-bfad33e1393c\") " pod="openshift-dns/node-resolver-k7r7n" Oct 30 18:37:39.599563 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.599067 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c876fc3-6980-4a99-aed5-7439ca3d6b69-trusted-ca\") pod \"console-operator-7cd56b8dbc-tp7sc\" (UID: \"0c876fc3-6980-4a99-aed5-7439ca3d6b69\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:39.599563 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.599092 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:39.599563 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.599119 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-sys\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.599563 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.599154 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-lib-modules\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.599563 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.599199 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8443eba-410b-4aea-94b7-f5fdabbb06e3-tmp\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.599563 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.599233 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q64j\" (UniqueName: \"kubernetes.io/projected/d8443eba-410b-4aea-94b7-f5fdabbb06e3-kube-api-access-9q64j\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.599563 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.599266 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-var-lib-kubelet\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.599563 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.599268 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d8443eba-410b-4aea-94b7-f5fdabbb06e3-tmp\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.599563 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.599312 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-conf-dir\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.599563 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.599337 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/13bb697e-8bff-4a30-9e1d-b9f3baac416e-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-gktbq\" (UID: \"13bb697e-8bff-4a30-9e1d-b9f3baac416e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq" Oct 30 18:37:39.599563 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.599380 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q87cx\" (UniqueName: \"kubernetes.io/projected/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-kube-api-access-q87cx\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:39.599563 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.599412 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjf45\" (UniqueName: \"kubernetes.io/projected/68788658-4afb-4757-acfa-7638cfb2488b-kube-api-access-mjf45\") pod \"network-check-source-6f78d59f99-d9fjm\" (UID: \"68788658-4afb-4757-acfa-7638cfb2488b\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-d9fjm" Oct 30 18:37:39.599563 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.599442 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.599563 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.599473 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z9kz\" (UniqueName: \"kubernetes.io/projected/b4a1d050-5a83-445c-a5a9-03ce4c64cc94-kube-api-access-5z9kz\") pod \"migrator-66df5d496b-7kvml\" (UID: \"b4a1d050-5a83-445c-a5a9-03ce4c64cc94\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-7kvml" Oct 30 18:37:39.599563 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.599515 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-etc-kubernetes\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.599563 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.599515 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/13bb697e-8bff-4a30-9e1d-b9f3baac416e-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-gktbq\" (UID: \"13bb697e-8bff-4a30-9e1d-b9f3baac416e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq" Oct 30 18:37:39.629767 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.629718 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.629852 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.629810 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Oct 30 18:37:39.631095 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.631080 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.631290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.631276 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.631369 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.631293 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.631369 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.631297 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.631369 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.631291 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 18:37:39.631369 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.631328 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.631369 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.631355 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.631369 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.631355 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 18:37:39.631566 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.631366 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 18:37:39.631566 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.631538 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Oct 30 18:37:39.631912 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.631863 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 18:37:39.631912 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.631899 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 18:37:39.632069 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.631926 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 18:37:39.632069 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.631972 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.632069 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.631977 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 18:37:39.632069 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.631985 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.632069 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.632043 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 18:37:39.635452 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.635430 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Oct 30 18:37:39.635522 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.635433 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.635522 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.635492 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.635522 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.635501 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Oct 30 18:37:39.635675 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.635540 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.635675 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.635550 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Oct 30 18:37:39.635675 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.635537 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.635675 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.635587 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 18:37:39.635675 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.635592 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.635675 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.635588 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 18:37:39.635675 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.635608 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.635675 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.635541 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.635675 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.635636 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 18:37:39.635675 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.635618 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.635675 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.635619 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 18:37:39.635675 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.635567 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.635675 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.635630 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.636083 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.635658 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.636083 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.635637 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.636083 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.635674 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.636083 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.635700 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.636083 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.635751 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-certificates\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:39.636290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.636150 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:37:39.636290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.636179 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-config-volume\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:39.636290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.636254 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/70781b52-75b2-450a-a058-b0d609659cda-ovnkube-config\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.636290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.636276 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/70781b52-75b2-450a-a058-b0d609659cda-ovnkube-script-lib\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.636411 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.636361 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/10a7e4f4-2efc-4803-b3c2-da8749031462-cni-binary-copy\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.636506 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.636489 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8a3c8868-93e2-4f20-910e-d2cd73bc1d5f-iptables-alerter-script\") pod \"iptables-alerter-p88j5\" (UID: \"8a3c8868-93e2-4f20-910e-d2cd73bc1d5f\") " pod="openshift-network-operator/iptables-alerter-p88j5" Oct 30 18:37:39.636600 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.636586 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-cni-binary-copy\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:37:39.636702 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.636686 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1340fb2d-5f77-4316-a156-f82e0cb1124d-nginx-conf\") pod \"networking-console-plugin-86c478bb88-dl96p\" (UID: \"1340fb2d-5f77-4316-a156-f82e0cb1124d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" Oct 30 18:37:39.636862 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.636845 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03157316-915c-412d-9a93-5edcf607a91f-service-ca-bundle\") pod \"periodic-gathering-lnkcs-v7pdj\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:37:39.637092 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.637072 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:37:39.637132 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.637107 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Oct 30 18:37:39.637248 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.637232 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d3be28-5482-48ea-a1be-629e8941236e-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:39.637773 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.637754 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-certificates\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:39.637857 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.637842 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1bec85c3-72c2-46dc-9f68-8e80f1478422-konnectivity-ca\") pod \"konnectivity-agent-w2wtr\" (UID: \"1bec85c3-72c2-46dc-9f68-8e80f1478422\") " pod="kube-system/konnectivity-agent-w2wtr" Oct 30 18:37:39.637942 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.637922 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a74aa745-ed96-4880-ad11-a31391e40f57-signing-cabundle\") pod \"service-ca-5fffc97bdf-rdt4m\" (UID: \"a74aa745-ed96-4880-ad11-a31391e40f57\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rdt4m" Oct 30 18:37:39.638015 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.637984 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03157316-915c-412d-9a93-5edcf607a91f-trusted-ca-bundle\") pod \"periodic-gathering-lnkcs-v7pdj\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:37:39.639099 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.639080 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/70781b52-75b2-450a-a058-b0d609659cda-env-overrides\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.639138 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.639119 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-daemon-config\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.639322 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.639305 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0c876fc3-6980-4a99-aed5-7439ca3d6b69-trusted-ca\") pod \"console-operator-7cd56b8dbc-tp7sc\" (UID: \"0c876fc3-6980-4a99-aed5-7439ca3d6b69\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:39.645605 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.645583 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d3be28-5482-48ea-a1be-629e8941236e-service-ca-bundle\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:39.645667 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.645609 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6a29b358-3cb7-49e6-85a9-6581a2f52bd1-serviceca\") pod \"node-ca-mxjc4\" (UID: \"6a29b358-3cb7-49e6-85a9-6581a2f52bd1\") " pod="openshift-image-registry/node-ca-mxjc4" Oct 30 18:37:39.645710 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.645686 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-trusted-ca\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:39.647923 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.647907 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-trusted-ca\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:39.664218 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.664199 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw7kf\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-kube-api-access-dw7kf\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:39.665394 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.665377 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhpbc\" (UniqueName: \"kubernetes.io/projected/10a7e4f4-2efc-4803-b3c2-da8749031462-kube-api-access-nhpbc\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.668873 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.668848 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twdrz\" (UniqueName: \"kubernetes.io/projected/9b5d4a9f-0410-49ba-ab07-c0c69e3fe45f-kube-api-access-twdrz\") pod \"kas-connection-checker-685895c7c9-kbclw\" (UID: \"9b5d4a9f-0410-49ba-ab07-c0c69e3fe45f\") " pod="kube-system/kas-connection-checker-685895c7c9-kbclw" Oct 30 18:37:39.669042 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.669021 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj9qm\" (UniqueName: \"kubernetes.io/projected/1704ffe1-d69c-4dd2-b7f6-5880bf1551f7-kube-api-access-wj9qm\") pod \"service-ca-operator-544d749c8c-b2rv6\" (UID: \"1704ffe1-d69c-4dd2-b7f6-5880bf1551f7\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6" Oct 30 18:37:39.669091 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.669041 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mpj6\" (UniqueName: \"kubernetes.io/projected/85fa03e6-458a-45e7-8c7b-a6820618a18a-kube-api-access-2mpj6\") pod \"migrator-66df5d496b-tsbpt\" (UID: \"85fa03e6-458a-45e7-8c7b-a6820618a18a\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-tsbpt" Oct 30 18:37:39.669205 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.669188 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zf4n\" (UniqueName: \"kubernetes.io/projected/b18682b8-f91a-4c27-87eb-82720ba4257f-kube-api-access-5zf4n\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:39.669280 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.669220 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w86l\" (UniqueName: \"kubernetes.io/projected/74153c90-cd72-43b8-85c5-aadb4d66a3e4-kube-api-access-8w86l\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:39.669375 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.669355 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr22p\" (UniqueName: \"kubernetes.io/projected/97294b18-463b-429d-8073-dc532ab0a046-kube-api-access-tr22p\") pod \"network-check-target-62vfn\" (UID: \"97294b18-463b-429d-8073-dc532ab0a046\") " pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:39.669565 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.669548 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-bound-sa-token\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:39.670753 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.670738 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5glk7\" (UniqueName: \"kubernetes.io/projected/13bb697e-8bff-4a30-9e1d-b9f3baac416e-kube-api-access-5glk7\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-gktbq\" (UID: \"13bb697e-8bff-4a30-9e1d-b9f3baac416e\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq" Oct 30 18:37:39.671993 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.671974 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flw7z\" (UniqueName: \"kubernetes.io/projected/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-kube-api-access-flw7z\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:39.673728 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.673699 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5tc8\" (UniqueName: \"kubernetes.io/projected/d8e44bd9-13d6-4e54-9d9b-bfad33e1393c-kube-api-access-l5tc8\") pod \"node-resolver-k7r7n\" (UID: \"d8e44bd9-13d6-4e54-9d9b-bfad33e1393c\") " pod="openshift-dns/node-resolver-k7r7n" Oct 30 18:37:39.674136 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.674112 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxznv\" (UniqueName: \"kubernetes.io/projected/25bce85b-a92c-4c39-a298-7b1881380069-kube-api-access-fxznv\") pod \"network-metrics-daemon-phngs\" (UID: \"25bce85b-a92c-4c39-a298-7b1881380069\") " pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:39.674415 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.674395 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbxh5\" (UniqueName: \"kubernetes.io/projected/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-kube-api-access-nbxh5\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:39.674513 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.674496 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4bc7\" (UniqueName: \"kubernetes.io/projected/03157316-915c-412d-9a93-5edcf607a91f-kube-api-access-g4bc7\") pod \"periodic-gathering-lnkcs-v7pdj\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:37:39.674560 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.674536 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lnmb\" (UniqueName: \"kubernetes.io/projected/14b832bf-1a7b-4539-811d-84a11d2b55b6-kube-api-access-9lnmb\") pod \"kas-connection-checker-685895c7c9-k4dpj\" (UID: \"14b832bf-1a7b-4539-811d-84a11d2b55b6\") " pod="kube-system/kas-connection-checker-685895c7c9-k4dpj" Oct 30 18:37:39.675210 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.675189 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5chv\" (UniqueName: \"kubernetes.io/projected/a74aa745-ed96-4880-ad11-a31391e40f57-kube-api-access-k5chv\") pod \"service-ca-5fffc97bdf-rdt4m\" (UID: \"a74aa745-ed96-4880-ad11-a31391e40f57\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rdt4m" Oct 30 18:37:39.675631 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.675608 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tww25\" (UniqueName: \"kubernetes.io/projected/70781b52-75b2-450a-a058-b0d609659cda-kube-api-access-tww25\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.676513 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.676491 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbb9x\" (UniqueName: \"kubernetes.io/projected/6a29b358-3cb7-49e6-85a9-6581a2f52bd1-kube-api-access-gbb9x\") pod \"node-ca-mxjc4\" (UID: \"6a29b358-3cb7-49e6-85a9-6581a2f52bd1\") " pod="openshift-image-registry/node-ca-mxjc4" Oct 30 18:37:39.677149 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.677131 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8pcp\" (UniqueName: \"kubernetes.io/projected/8a3c8868-93e2-4f20-910e-d2cd73bc1d5f-kube-api-access-v8pcp\") pod \"iptables-alerter-p88j5\" (UID: \"8a3c8868-93e2-4f20-910e-d2cd73bc1d5f\") " pod="openshift-network-operator/iptables-alerter-p88j5" Oct 30 18:37:39.677197 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.677161 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8j7j\" (UniqueName: \"kubernetes.io/projected/6050488d-f6ab-4bab-b2b2-90d30f64bab4-kube-api-access-d8j7j\") pod \"kas-connection-checker-685895c7c9-mzqcg\" (UID: \"6050488d-f6ab-4bab-b2b2-90d30f64bab4\") " pod="kube-system/kas-connection-checker-685895c7c9-mzqcg" Oct 30 18:37:39.677979 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.677965 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4q88\" (UniqueName: \"kubernetes.io/projected/00d3be28-5482-48ea-a1be-629e8941236e-kube-api-access-k4q88\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:39.680550 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.680534 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q87cx\" (UniqueName: \"kubernetes.io/projected/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-kube-api-access-q87cx\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:39.680590 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.680566 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7n2h\" (UniqueName: \"kubernetes.io/projected/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-kube-api-access-n7n2h\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:37:39.680669 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.680655 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t24cp\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-kube-api-access-t24cp\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:39.682264 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.682247 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjf45\" (UniqueName: \"kubernetes.io/projected/68788658-4afb-4757-acfa-7638cfb2488b-kube-api-access-mjf45\") pod \"network-check-source-6f78d59f99-d9fjm\" (UID: \"68788658-4afb-4757-acfa-7638cfb2488b\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-d9fjm" Oct 30 18:37:39.687164 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.683141 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gqkz\" (UniqueName: \"kubernetes.io/projected/0c876fc3-6980-4a99-aed5-7439ca3d6b69-kube-api-access-5gqkz\") pod \"console-operator-7cd56b8dbc-tp7sc\" (UID: \"0c876fc3-6980-4a99-aed5-7439ca3d6b69\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:39.687164 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.686182 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-czdz5\"" Oct 30 18:37:39.687164 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.686266 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 18:37:39.687164 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.686318 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hlrng\"" Oct 30 18:37:39.687164 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.686458 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4rg2h\"" Oct 30 18:37:39.687164 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.686546 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-w4fd8\"" Oct 30 18:37:39.687164 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.686573 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 18:37:39.687164 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.686640 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-gsdbf\"" Oct 30 18:37:39.687164 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.686773 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 18:37:39.687164 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.686870 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-smcdv\"" Oct 30 18:37:39.687164 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.686903 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Oct 30 18:37:39.687164 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.686988 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gh5g8\"" Oct 30 18:37:39.687164 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.687111 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1bec85c3-72c2-46dc-9f68-8e80f1478422-agent-certs\") pod \"konnectivity-agent-w2wtr\" (UID: \"1bec85c3-72c2-46dc-9f68-8e80f1478422\") " pod="kube-system/konnectivity-agent-w2wtr" Oct 30 18:37:39.687164 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.687125 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Oct 30 18:37:39.687844 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.687155 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Oct 30 18:37:39.687844 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.687247 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-fv5kc\"" Oct 30 18:37:39.687844 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.687368 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Oct 30 18:37:39.687844 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.687374 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-j99tc\"" Oct 30 18:37:39.687844 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.687411 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-s6dvj\"" Oct 30 18:37:39.687844 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.687442 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-lp85d\"" Oct 30 18:37:39.687844 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.687472 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f1cae505-4cef-46e5-85cc-cf8b410e654a-original-pull-secret\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:39.687844 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.687566 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 18:37:39.687844 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.687572 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9cgrs\"" Oct 30 18:37:39.687844 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.687617 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Oct 30 18:37:39.687844 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.687627 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7kh4z\"" Oct 30 18:37:39.687844 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.687715 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q64j\" (UniqueName: \"kubernetes.io/projected/d8443eba-410b-4aea-94b7-f5fdabbb06e3-kube-api-access-9q64j\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.687844 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.687729 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 18:37:39.687844 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.687767 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 18:37:39.688495 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.687880 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-m4f62\"" Oct 30 18:37:39.688495 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.687946 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6k6g\" (UniqueName: \"kubernetes.io/projected/2297f719-7e19-4de8-b59c-f75885db9cc6-kube-api-access-v6k6g\") pod \"volume-data-source-validator-58c9f5f46d-nprgd\" (UID: \"2297f719-7e19-4de8-b59c-f75885db9cc6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-nprgd" Oct 30 18:37:39.688495 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.687958 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-f6d2v\"" Oct 30 18:37:39.688495 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.687255 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 18:37:39.688495 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.688062 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 18:37:39.688495 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.688116 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lfj5d\"" Oct 30 18:37:39.688495 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.687125 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jqqcl\"" Oct 30 18:37:39.688495 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.688255 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Oct 30 18:37:39.688495 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.688274 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftcrx\" (UniqueName: \"kubernetes.io/projected/b7ad340e-8e80-4f55-a29b-e8d501946cb5-kube-api-access-ftcrx\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:37:39.688495 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.688292 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-t9ljp\"" Oct 30 18:37:39.688495 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.688315 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-4frj9\"" Oct 30 18:37:39.688495 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.688327 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-bound-sa-token\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:39.688495 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.688351 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-q6kg8\"" Oct 30 18:37:39.688495 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.688465 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/542d052a-724f-4d98-9be7-7765a8c1d80a-cert\") pod \"ingress-canary-rbmpn\" (UID: \"542d052a-724f-4d98-9be7-7765a8c1d80a\") " pod="openshift-ingress-canary/ingress-canary-rbmpn" Oct 30 18:37:39.688886 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.688868 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkd85\" (UniqueName: \"kubernetes.io/projected/542d052a-724f-4d98-9be7-7765a8c1d80a-kube-api-access-fkd85\") pod \"ingress-canary-rbmpn\" (UID: \"542d052a-724f-4d98-9be7-7765a8c1d80a\") " pod="openshift-ingress-canary/ingress-canary-rbmpn" Oct 30 18:37:39.688986 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.688971 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a74aa745-ed96-4880-ad11-a31391e40f57-signing-key\") pod \"service-ca-5fffc97bdf-rdt4m\" (UID: \"a74aa745-ed96-4880-ad11-a31391e40f57\") " pod="openshift-service-ca/service-ca-5fffc97bdf-rdt4m" Oct 30 18:37:39.689049 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.689038 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-installation-pull-secrets\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:39.689262 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.689248 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/70781b52-75b2-450a-a058-b0d609659cda-ovn-node-metrics-cert\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.694287 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.694269 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-image-registry-private-configuration\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:39.694393 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.694377 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/00d3be28-5482-48ea-a1be-629e8941236e-serving-cert\") pod \"insights-operator-7c4986d449-q9rgp\" (UID: \"00d3be28-5482-48ea-a1be-629e8941236e\") " pod="openshift-insights/insights-operator-7c4986d449-q9rgp" Oct 30 18:37:39.695837 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.695821 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-installation-pull-secrets\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:39.696415 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.696397 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-image-registry-private-configuration\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:39.696645 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.696630 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z9kz\" (UniqueName: \"kubernetes.io/projected/b4a1d050-5a83-445c-a5a9-03ce4c64cc94-kube-api-access-5z9kz\") pod \"migrator-66df5d496b-7kvml\" (UID: \"b4a1d050-5a83-445c-a5a9-03ce4c64cc94\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-7kvml" Oct 30 18:37:39.700866 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.700850 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-run-netns\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.700906 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.700874 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-host\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.700906 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.700889 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-etc-selinux\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:37:39.700906 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.700903 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:39.701049 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.700919 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-sysctl-conf\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.701049 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.700948 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-host\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.701049 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.700973 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-run-netns\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.701049 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:39.700987 5054 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:37:39.701049 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:39.700997 5054 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5df4b54dc9-8pdtm: secret "image-registry-tls" not found Oct 30 18:37:39.701049 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701027 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701055 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-sys\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:39.701067 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls podName:7fcdea8f-3b0f-42c5-bb3d-b2266b046491 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:40.20105398 +0000 UTC m=+1.929432693 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls") pod "image-registry-5df4b54dc9-8pdtm" (UID: "7fcdea8f-3b0f-42c5-bb3d-b2266b046491") : secret "image-registry-tls" not found Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701085 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-lib-modules\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701099 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-sys\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701101 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-var-lib-kubelet\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701129 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-conf-dir\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701136 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-var-lib-kubelet\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701154 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:39.701164 5054 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701172 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-etc-selinux\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701176 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-etc-kubernetes\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701203 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-cnibin\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701216 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-sysctl-conf\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:39.701180 5054 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649485556b-7x5gc: secret "image-registry-tls" not found Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701233 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701226 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-crio-socket\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701263 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-crio-socket\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701216 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-conf-dir\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:39.701281 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls podName:d9b465b9-97b6-47e2-8732-ed29b8f24ed0 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:40.201267837 +0000 UTC m=+1.929646550 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls") pod "image-registry-649485556b-7x5gc" (UID: "d9b465b9-97b6-47e2-8732-ed29b8f24ed0") : secret "image-registry-tls" not found Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701278 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-lib-modules\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701294 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-cnibin\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701216 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-etc-kubernetes\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.701302 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701300 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-cni-netd\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701323 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-cni-netd\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701336 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-cni-dir\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701361 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-hostroot\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701388 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701399 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-cni-dir\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701412 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701444 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-hostroot\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701450 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-system-cni-dir\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:39.701473 5054 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701476 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-sys-fs\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701481 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-system-cni-dir\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701500 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-run-ovn\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701522 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-kubernetes\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:39.701541 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls podName:74153c90-cd72-43b8-85c5-aadb4d66a3e4 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:40.201528028 +0000 UTC m=+1.929906745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-cf8qt" (UID: "74153c90-cd72-43b8-85c5-aadb4d66a3e4") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701545 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-sys-fs\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701572 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-kubernetes\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701574 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701588 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-run-ovn\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:39.701595 5054 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701611 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-run-systemd\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:39.701618 5054 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:39.701635 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls podName:447f2b64-f13e-48cc-94b9-e80df4a9d4f3 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:40.201625347 +0000 UTC m=+1.930004044 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls") pod "dns-default-ltkzx" (UID: "447f2b64-f13e-48cc-94b9-e80df4a9d4f3") : secret "dns-default-metrics-tls" not found Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:39.701650 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls podName:3ccc9d41-f2e7-4b0d-a064-cb3e5549049a nodeName:}" failed. No retries permitted until 2026-10-30 18:37:40.201639538 +0000 UTC m=+1.930018231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bscj4" (UID: "3ccc9d41-f2e7-4b0d-a064-cb3e5549049a") : secret "insights-runtime-extractor-tls" not found Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701654 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-run-systemd\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701663 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-var-lib-openvswitch\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701680 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f1cae505-4cef-46e5-85cc-cf8b410e654a-dbus\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701696 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701708 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-var-lib-openvswitch\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701711 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701738 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d8e44bd9-13d6-4e54-9d9b-bfad33e1393c-hosts-file\") pod \"node-resolver-k7r7n\" (UID: \"d8e44bd9-13d6-4e54-9d9b-bfad33e1393c\") " pod="openshift-dns/node-resolver-k7r7n" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701752 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-node-log\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:39.701755 5054 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701770 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-run-ovn-kubernetes\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:39.701784 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls podName:b18682b8-f91a-4c27-87eb-82720ba4257f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:40.201774557 +0000 UTC m=+1.930153255 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6mt5b" (UID: "b18682b8-f91a-4c27-87eb-82720ba4257f") : secret "samples-operator-tls" not found Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701783 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f1cae505-4cef-46e5-85cc-cf8b410e654a-dbus\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701794 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d8e44bd9-13d6-4e54-9d9b-bfad33e1393c-hosts-file\") pod \"node-resolver-k7r7n\" (UID: \"d8e44bd9-13d6-4e54-9d9b-bfad33e1393c\") " pod="openshift-dns/node-resolver-k7r7n" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701801 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-node-log\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701809 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-etc-openvswitch\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701802 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-run-ovn-kubernetes\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:39.701819 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle podName:8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:40.201808313 +0000 UTC m=+1.930187031 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle") pod "router-default-6c975679c5-x5lg8" (UID: "8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7") : configmap references non-existent config key: service-ca.crt Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701845 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-etc-openvswitch\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701855 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701878 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701896 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-var-lib-cni-bin\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701912 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-system-cni-dir\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701929 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-modprobe-d\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701898 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701945 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-sysctl-d\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701931 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-var-lib-cni-bin\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701963 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-run\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701979 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.701997 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-var-lib-cni-multus\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702034 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-modprobe-d\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702036 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-run-multus-certs\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702043 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-run\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702060 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66d0cf78-7aa4-40f2-ab29-dd33a5c73979-system-cni-dir\") pod \"multus-additional-cni-plugins-7dkzl\" (UID: \"66d0cf78-7aa4-40f2-ab29-dd33a5c73979\") " pod="openshift-multus/multus-additional-cni-plugins-7dkzl" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702059 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-slash\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702077 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-sysctl-d\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702087 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-run-openvswitch\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702090 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-var-lib-cni-multus\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702096 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-slash\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702057 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702096 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-run-multus-certs\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702113 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-socket-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702120 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-run-openvswitch\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702137 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a3c8868-93e2-4f20-910e-d2cd73bc1d5f-host-slash\") pod \"iptables-alerter-p88j5\" (UID: \"8a3c8868-93e2-4f20-910e-d2cd73bc1d5f\") " pod="openshift-network-operator/iptables-alerter-p88j5" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702147 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-socket-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702156 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-kubelet\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702176 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-device-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702186 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a3c8868-93e2-4f20-910e-d2cd73bc1d5f-host-slash\") pod \"iptables-alerter-p88j5\" (UID: \"8a3c8868-93e2-4f20-910e-d2cd73bc1d5f\") " pod="openshift-network-operator/iptables-alerter-p88j5" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702206 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs\") pod \"network-metrics-daemon-phngs\" (UID: \"25bce85b-a92c-4c39-a298-7b1881380069\") " pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702215 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-kubelet\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702236 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-log-socket\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.702212 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702251 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-device-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702266 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-run-k8s-cni-cncf-io\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702289 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-log-socket\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702298 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-cni-bin\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:39.702318 5054 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702322 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-systemd-units\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702324 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-run-k8s-cni-cncf-io\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702345 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-sysconfig\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702354 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-host-cni-bin\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702360 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/70781b52-75b2-450a-a058-b0d609659cda-systemd-units\") pod \"ovnkube-node-4hsc7\" (UID: \"70781b52-75b2-450a-a058-b0d609659cda\") " pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:39.702377 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs podName:25bce85b-a92c-4c39-a298-7b1881380069 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:40.20236307 +0000 UTC m=+1.930741764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs") pod "network-metrics-daemon-phngs" (UID: "25bce85b-a92c-4c39-a298-7b1881380069") : secret "metrics-daemon-secret" not found Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702392 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-sysconfig\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702396 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-cnibin\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702418 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702425 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-cnibin\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702430 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-metrics-certs\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702458 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-socket-dir-parent\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702473 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-run-netns\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702487 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-registration-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702506 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-dl96p\" (UID: \"1340fb2d-5f77-4316-a156-f82e0cb1124d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702519 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-host-run-netns\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702533 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/10a7e4f4-2efc-4803-b3c2-da8749031462-multus-socket-dir-parent\") pod \"multus-928bp\" (UID: \"10a7e4f4-2efc-4803-b3c2-da8749031462\") " pod="openshift-multus/multus-928bp" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702557 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b7ad340e-8e80-4f55-a29b-e8d501946cb5-registration-dir\") pod \"aws-ebs-csi-driver-node-2g5jv\" (UID: \"b7ad340e-8e80-4f55-a29b-e8d501946cb5\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2g5jv" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:39.702578 5054 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702600 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-systemd\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:39.702615 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert podName:1340fb2d-5f77-4316-a156-f82e0cb1124d nodeName:}" failed. No retries permitted until 2026-10-30 18:37:40.202605488 +0000 UTC m=+1.930984216 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-dl96p" (UID: "1340fb2d-5f77-4316-a156-f82e0cb1124d") : secret "networking-console-plugin-cert" not found Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702578 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-etc-systemd\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702644 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-var-lib-kubelet\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702670 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f1cae505-4cef-46e5-85cc-cf8b410e654a-kubelet-config\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702694 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a29b358-3cb7-49e6-85a9-6581a2f52bd1-host\") pod \"node-ca-mxjc4\" (UID: \"6a29b358-3cb7-49e6-85a9-6581a2f52bd1\") " pod="openshift-image-registry/node-ca-mxjc4" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702740 5054 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702764 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a29b358-3cb7-49e6-85a9-6581a2f52bd1-host\") pod \"node-ca-mxjc4\" (UID: \"6a29b358-3cb7-49e6-85a9-6581a2f52bd1\") " pod="openshift-image-registry/node-ca-mxjc4" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702757 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8443eba-410b-4aea-94b7-f5fdabbb06e3-var-lib-kubelet\") pod \"tuned-fbfdv\" (UID: \"d8443eba-410b-4aea-94b7-f5fdabbb06e3\") " pod="openshift-cluster-node-tuning-operator/tuned-fbfdv" Oct 30 18:37:39.704348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.702757 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f1cae505-4cef-46e5-85cc-cf8b410e654a-kubelet-config\") pod \"global-pull-secret-syncer-pxndl\" (UID: \"f1cae505-4cef-46e5-85cc-cf8b410e654a\") " pod="kube-system/global-pull-secret-syncer-pxndl" Oct 30 18:37:39.705239 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.704805 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-metrics-certs\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:39.829165 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:39.829141 5054 scope.go:122] "RemoveContainer" containerID="5b64ecda12ea0b0a79e9aae4b5a3057c653329e1c1e98c72fb2411e0caca677e" Oct 30 18:37:40.206585 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:40.206510 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs\") pod \"network-metrics-daemon-phngs\" (UID: \"25bce85b-a92c-4c39-a298-7b1881380069\") " pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:40.206585 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:40.206557 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-dl96p\" (UID: \"1340fb2d-5f77-4316-a156-f82e0cb1124d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" Oct 30 18:37:40.206585 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:40.206581 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:40.206793 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:40.206610 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:40.206793 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:40.206639 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:40.206793 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:40.206657 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:40.206793 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:40.206663 5054 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:37:40.206793 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:40.206735 5054 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:37:40.206793 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:40.206742 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert podName:1340fb2d-5f77-4316-a156-f82e0cb1124d nodeName:}" failed. No retries permitted until 2026-10-30 18:37:41.206722359 +0000 UTC m=+2.935101074 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-dl96p" (UID: "1340fb2d-5f77-4316-a156-f82e0cb1124d") : secret "networking-console-plugin-cert" not found Oct 30 18:37:40.206793 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:40.206746 5054 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:37:40.206793 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:40.206755 5054 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Oct 30 18:37:40.206793 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:40.206775 5054 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:37:40.206793 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:40.206792 5054 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5df4b54dc9-8pdtm: secret "image-registry-tls" not found Oct 30 18:37:40.206793 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:40.206797 5054 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:37:40.207164 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:40.206780 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls podName:447f2b64-f13e-48cc-94b9-e80df4a9d4f3 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:41.206770039 +0000 UTC m=+2.935148746 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls") pod "dns-default-ltkzx" (UID: "447f2b64-f13e-48cc-94b9-e80df4a9d4f3") : secret "dns-default-metrics-tls" not found Oct 30 18:37:40.207164 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:40.206840 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs podName:25bce85b-a92c-4c39-a298-7b1881380069 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:41.206829352 +0000 UTC m=+2.935208046 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs") pod "network-metrics-daemon-phngs" (UID: "25bce85b-a92c-4c39-a298-7b1881380069") : secret "metrics-daemon-secret" not found Oct 30 18:37:40.207164 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:40.206751 5054 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649485556b-7x5gc: secret "image-registry-tls" not found Oct 30 18:37:40.207164 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:40.206871 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:40.207164 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:40.206895 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls podName:d9b465b9-97b6-47e2-8732-ed29b8f24ed0 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:41.20688273 +0000 UTC m=+2.935261441 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls") pod "image-registry-649485556b-7x5gc" (UID: "d9b465b9-97b6-47e2-8732-ed29b8f24ed0") : secret "image-registry-tls" not found Oct 30 18:37:40.207164 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:40.206905 5054 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 18:37:40.207164 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:40.206914 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls podName:7fcdea8f-3b0f-42c5-bb3d-b2266b046491 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:41.206908636 +0000 UTC m=+2.935287356 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls") pod "image-registry-5df4b54dc9-8pdtm" (UID: "7fcdea8f-3b0f-42c5-bb3d-b2266b046491") : secret "image-registry-tls" not found Oct 30 18:37:40.207164 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:40.206928 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls podName:74153c90-cd72-43b8-85c5-aadb4d66a3e4 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:41.206921436 +0000 UTC m=+2.935300129 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-cf8qt" (UID: "74153c90-cd72-43b8-85c5-aadb4d66a3e4") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:37:40.207164 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:40.206947 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls podName:3ccc9d41-f2e7-4b0d-a064-cb3e5549049a nodeName:}" failed. No retries permitted until 2026-10-30 18:37:41.206936566 +0000 UTC m=+2.935315259 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bscj4" (UID: "3ccc9d41-f2e7-4b0d-a064-cb3e5549049a") : secret "insights-runtime-extractor-tls" not found Oct 30 18:37:40.207164 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:40.206972 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:40.207164 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:40.206989 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:40.207164 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:40.207067 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle podName:8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:41.207060273 +0000 UTC m=+2.935439008 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle") pod "router-default-6c975679c5-x5lg8" (UID: "8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7") : configmap references non-existent config key: service-ca.crt Oct 30 18:37:40.207164 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:40.207094 5054 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:37:40.207164 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:40.207125 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls podName:b18682b8-f91a-4c27-87eb-82720ba4257f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:41.2071182 +0000 UTC m=+2.935496921 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6mt5b" (UID: "b18682b8-f91a-4c27-87eb-82720ba4257f") : secret "samples-operator-tls" not found Oct 30 18:37:40.480510 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:40.480438 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 18:37:40.480510 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:40.480490 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" event={"ID":"0c876fc3-6980-4a99-aed5-7439ca3d6b69","Type":"ContainerStarted","Data":"a6b7b0e3828e7e21cd38fc5e7b7b1d64909a87fe5f3cb19d4c97ee33a48cdc5c"} Oct 30 18:37:41.215498 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:41.215462 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:41.215498 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:41.215500 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:41.215723 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:41.215524 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:41.215723 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:41.215546 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:41.215723 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:41.215564 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:41.215723 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:41.215611 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs\") pod \"network-metrics-daemon-phngs\" (UID: \"25bce85b-a92c-4c39-a298-7b1881380069\") " pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:41.215723 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:41.215636 5054 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:37:41.215723 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:41.215650 5054 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 18:37:41.215723 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:41.215643 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-dl96p\" (UID: \"1340fb2d-5f77-4316-a156-f82e0cb1124d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" Oct 30 18:37:41.215723 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:41.215699 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle podName:8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:43.215679088 +0000 UTC m=+4.944057796 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle") pod "router-default-6c975679c5-x5lg8" (UID: "8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7") : configmap references non-existent config key: service-ca.crt Oct 30 18:37:41.215723 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:41.215712 5054 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:37:41.215723 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:41.215717 5054 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:37:41.216039 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:41.215732 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls podName:74153c90-cd72-43b8-85c5-aadb4d66a3e4 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:43.215721259 +0000 UTC m=+4.944099953 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-cf8qt" (UID: "74153c90-cd72-43b8-85c5-aadb4d66a3e4") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:37:41.216039 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:41.215747 5054 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Oct 30 18:37:41.216039 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:41.215753 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls podName:3ccc9d41-f2e7-4b0d-a064-cb3e5549049a nodeName:}" failed. No retries permitted until 2026-10-30 18:37:43.215742711 +0000 UTC m=+4.944121511 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bscj4" (UID: "3ccc9d41-f2e7-4b0d-a064-cb3e5549049a") : secret "insights-runtime-extractor-tls" not found Oct 30 18:37:41.216039 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:41.215636 5054 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:37:41.216039 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:41.215773 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls podName:b18682b8-f91a-4c27-87eb-82720ba4257f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:43.215765324 +0000 UTC m=+4.944144018 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6mt5b" (UID: "b18682b8-f91a-4c27-87eb-82720ba4257f") : secret "samples-operator-tls" not found Oct 30 18:37:41.216039 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:41.215795 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls podName:447f2b64-f13e-48cc-94b9-e80df4a9d4f3 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:43.215789184 +0000 UTC m=+4.944167878 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls") pod "dns-default-ltkzx" (UID: "447f2b64-f13e-48cc-94b9-e80df4a9d4f3") : secret "dns-default-metrics-tls" not found Oct 30 18:37:41.216039 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:41.215806 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert podName:1340fb2d-5f77-4316-a156-f82e0cb1124d nodeName:}" failed. No retries permitted until 2026-10-30 18:37:43.215801685 +0000 UTC m=+4.944180379 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-dl96p" (UID: "1340fb2d-5f77-4316-a156-f82e0cb1124d") : secret "networking-console-plugin-cert" not found Oct 30 18:37:41.216039 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:41.215817 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs podName:25bce85b-a92c-4c39-a298-7b1881380069 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:43.21581205 +0000 UTC m=+4.944190743 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs") pod "network-metrics-daemon-phngs" (UID: "25bce85b-a92c-4c39-a298-7b1881380069") : secret "metrics-daemon-secret" not found Oct 30 18:37:41.216039 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:41.215848 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:41.216039 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:41.215869 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:41.216039 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:41.215953 5054 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:37:41.216039 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:41.215962 5054 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-649485556b-7x5gc: secret "image-registry-tls" not found Oct 30 18:37:41.216039 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:41.215969 5054 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:37:41.216039 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:41.215977 5054 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5df4b54dc9-8pdtm: secret "image-registry-tls" not found Oct 30 18:37:41.216039 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:41.215990 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls podName:d9b465b9-97b6-47e2-8732-ed29b8f24ed0 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:43.215983823 +0000 UTC m=+4.944362516 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls") pod "image-registry-649485556b-7x5gc" (UID: "d9b465b9-97b6-47e2-8732-ed29b8f24ed0") : secret "image-registry-tls" not found Oct 30 18:37:41.216039 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:41.216019 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls podName:7fcdea8f-3b0f-42c5-bb3d-b2266b046491 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:43.215995182 +0000 UTC m=+4.944373876 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls") pod "image-registry-5df4b54dc9-8pdtm" (UID: "7fcdea8f-3b0f-42c5-bb3d-b2266b046491") : secret "image-registry-tls" not found Oct 30 18:37:41.326176 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:41.326149 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k7r7n_d8e44bd9-13d6-4e54-9d9b-bfad33e1393c/dns-node-resolver/0.log" Oct 30 18:37:41.910663 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:41.910637 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mxjc4_6a29b358-3cb7-49e6-85a9-6581a2f52bd1/node-ca/0.log" Oct 30 18:37:42.706480 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:42.706448 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rbmpn_542d052a-724f-4d98-9be7-7765a8c1d80a/serve-healthcheck-canary/0.log" Oct 30 18:37:42.911240 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:42.911208 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-7kvml_b4a1d050-5a83-445c-a5a9-03ce4c64cc94/migrator/0.log" Oct 30 18:37:43.043330 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.043268 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:43.043472 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.043364 5054 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:37:43.048015 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.047985 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-tp7sc" Oct 30 18:37:43.128988 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.128957 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-tsbpt_85fa03e6-458a-45e7-8c7b-a6820618a18a/migrator/0.log" Oct 30 18:37:43.180314 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.180290 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-w2wtr" Oct 30 18:37:43.234982 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.234953 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-dl96p\" (UID: \"1340fb2d-5f77-4316-a156-f82e0cb1124d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" Oct 30 18:37:43.235154 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.234994 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:43.235154 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.235045 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:43.235154 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.235073 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:43.235154 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.235089 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:43.235154 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:43.235124 5054 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:37:43.235406 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:43.235168 5054 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:37:43.235406 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:43.235199 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert podName:1340fb2d-5f77-4316-a156-f82e0cb1124d nodeName:}" failed. No retries permitted until 2026-10-30 18:37:47.235181597 +0000 UTC m=+8.963560291 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-dl96p" (UID: "1340fb2d-5f77-4316-a156-f82e0cb1124d") : secret "networking-console-plugin-cert" not found Oct 30 18:37:43.235406 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:43.235203 5054 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:37:43.235406 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:43.235224 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls podName:447f2b64-f13e-48cc-94b9-e80df4a9d4f3 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:47.235208842 +0000 UTC m=+8.963587537 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls") pod "dns-default-ltkzx" (UID: "447f2b64-f13e-48cc-94b9-e80df4a9d4f3") : secret "dns-default-metrics-tls" not found Oct 30 18:37:43.235406 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:43.235269 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls podName:74153c90-cd72-43b8-85c5-aadb4d66a3e4 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:47.235252362 +0000 UTC m=+8.963631071 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-cf8qt" (UID: "74153c90-cd72-43b8-85c5-aadb4d66a3e4") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:37:43.235406 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.235296 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:43.235406 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.235344 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:43.235406 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.235389 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:43.235791 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:43.235458 5054 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 18:37:43.235791 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.235460 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs\") pod \"network-metrics-daemon-phngs\" (UID: \"25bce85b-a92c-4c39-a298-7b1881380069\") " pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:43.235791 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:43.235519 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls podName:3ccc9d41-f2e7-4b0d-a064-cb3e5549049a nodeName:}" failed. No retries permitted until 2026-10-30 18:37:47.235503658 +0000 UTC m=+8.963882365 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bscj4" (UID: "3ccc9d41-f2e7-4b0d-a064-cb3e5549049a") : secret "insights-runtime-extractor-tls" not found Oct 30 18:37:43.235791 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:43.235559 5054 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:37:43.235791 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:43.235583 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle podName:8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:47.235571577 +0000 UTC m=+8.963950270 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle") pod "router-default-6c975679c5-x5lg8" (UID: "8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7") : configmap references non-existent config key: service-ca.crt Oct 30 18:37:43.235791 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:37:43.235601 5054 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls podName:b18682b8-f91a-4c27-87eb-82720ba4257f nodeName:}" failed. No retries permitted until 2026-10-30 18:37:47.235591267 +0000 UTC m=+8.963969961 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-6mt5b" (UID: "b18682b8-f91a-4c27-87eb-82720ba4257f") : secret "samples-operator-tls" not found Oct 30 18:37:43.237514 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.237493 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25bce85b-a92c-4c39-a298-7b1881380069-metrics-certs\") pod \"network-metrics-daemon-phngs\" (UID: \"25bce85b-a92c-4c39-a298-7b1881380069\") " pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:43.237616 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.237493 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls\") pod \"image-registry-5df4b54dc9-8pdtm\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:43.237616 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.237537 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls\") pod \"image-registry-649485556b-7x5gc\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:43.312831 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.312765 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-gktbq_13bb697e-8bff-4a30-9e1d-b9f3baac416e/kube-storage-version-migrator-operator/0.log" Oct 30 18:37:43.386273 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.386249 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:43.430610 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.430580 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:43.434808 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.434791 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-phngs" Oct 30 18:37:43.437674 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.437554 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:43.872988 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.872956 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-649485556b-7x5gc"] Oct 30 18:37:43.879750 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:43.879719 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9b465b9_97b6_47e2_8732_ed29b8f24ed0.slice/crio-e33b01a9db9bbb98f5ad302eaf29eac84dc00c5c47636d085bb9a314fe877f5a WatchSource:0}: Error finding container e33b01a9db9bbb98f5ad302eaf29eac84dc00c5c47636d085bb9a314fe877f5a: Status 404 returned error can't find the container with id e33b01a9db9bbb98f5ad302eaf29eac84dc00c5c47636d085bb9a314fe877f5a Oct 30 18:37:43.884140 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.884122 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-phngs"] Oct 30 18:37:43.887872 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.887855 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5df4b54dc9-8pdtm"] Oct 30 18:37:43.890453 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:43.890428 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25bce85b_a92c_4c39_a298_7b1881380069.slice/crio-f698cd776c81f09574dfa5a3c32563aa86eacce80785afee8e5e8286c75cf28d WatchSource:0}: Error finding container f698cd776c81f09574dfa5a3c32563aa86eacce80785afee8e5e8286c75cf28d: Status 404 returned error can't find the container with id f698cd776c81f09574dfa5a3c32563aa86eacce80785afee8e5e8286c75cf28d Oct 30 18:37:43.892395 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:43.892378 5054 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:37:43.908017 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:43.907982 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fcdea8f_3b0f_42c5_bb3d_b2266b046491.slice/crio-3dbf5737a4e81ce7dffe8835c803744bc14134b845b162a0485f14815e3b6f0f WatchSource:0}: Error finding container 3dbf5737a4e81ce7dffe8835c803744bc14134b845b162a0485f14815e3b6f0f: Status 404 returned error can't find the container with id 3dbf5737a4e81ce7dffe8835c803744bc14134b845b162a0485f14815e3b6f0f Oct 30 18:37:44.499910 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:44.499878 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-649485556b-7x5gc" event={"ID":"d9b465b9-97b6-47e2-8732-ed29b8f24ed0","Type":"ContainerStarted","Data":"5ac2d38334da6fc2906b90398de622fbd49104f8b8a3ea6afbf3c8dfb2d22f48"} Oct 30 18:37:44.500408 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:44.499932 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-649485556b-7x5gc" event={"ID":"d9b465b9-97b6-47e2-8732-ed29b8f24ed0","Type":"ContainerStarted","Data":"e33b01a9db9bbb98f5ad302eaf29eac84dc00c5c47636d085bb9a314fe877f5a"} Oct 30 18:37:44.501287 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:44.501266 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" event={"ID":"7fcdea8f-3b0f-42c5-bb3d-b2266b046491","Type":"ContainerStarted","Data":"31426b35dccce5ef8e1411971efb6b5ab9c9b15ddc6fdc8bc01ed75a905773af"} Oct 30 18:37:44.501387 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:44.501302 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" event={"ID":"7fcdea8f-3b0f-42c5-bb3d-b2266b046491","Type":"ContainerStarted","Data":"3dbf5737a4e81ce7dffe8835c803744bc14134b845b162a0485f14815e3b6f0f"} Oct 30 18:37:44.502353 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:44.502328 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-phngs" event={"ID":"25bce85b-a92c-4c39-a298-7b1881380069","Type":"ContainerStarted","Data":"f698cd776c81f09574dfa5a3c32563aa86eacce80785afee8e5e8286c75cf28d"} Oct 30 18:37:44.856643 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:44.856568 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:37:45.507669 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:45.507634 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-phngs" event={"ID":"25bce85b-a92c-4c39-a298-7b1881380069","Type":"ContainerStarted","Data":"2f1930e48524085b6b0a79fe54c85fd8cbefe4e585923ee34a1c9214f99645d6"} Oct 30 18:37:45.507669 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:45.507662 5054 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:37:45.508160 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:45.507698 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-phngs" event={"ID":"25bce85b-a92c-4c39-a298-7b1881380069","Type":"ContainerStarted","Data":"9d4a0c10149ad54c9b90a526fff817dfb1eb5ef852d9bbb27fa488f7bb260452"} Oct 30 18:37:47.273127 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.273090 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-dl96p\" (UID: \"1340fb2d-5f77-4316-a156-f82e0cb1124d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" Oct 30 18:37:47.273762 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.273140 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:47.273762 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.273157 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:47.273762 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.273212 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:47.273762 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.273248 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:47.273762 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.273282 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:47.274090 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.274032 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7-service-ca-bundle\") pod \"router-default-6c975679c5-x5lg8\" (UID: \"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7\") " pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:47.275831 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.275797 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/447f2b64-f13e-48cc-94b9-e80df4a9d4f3-metrics-tls\") pod \"dns-default-ltkzx\" (UID: \"447f2b64-f13e-48cc-94b9-e80df4a9d4f3\") " pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:47.275831 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.275804 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3ccc9d41-f2e7-4b0d-a064-cb3e5549049a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bscj4\" (UID: \"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a\") " pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:47.275963 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.275868 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/74153c90-cd72-43b8-85c5-aadb4d66a3e4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-cf8qt\" (UID: \"74153c90-cd72-43b8-85c5-aadb4d66a3e4\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:47.276221 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.276202 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1340fb2d-5f77-4316-a156-f82e0cb1124d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-dl96p\" (UID: \"1340fb2d-5f77-4316-a156-f82e0cb1124d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" Oct 30 18:37:47.276265 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.276202 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b18682b8-f91a-4c27-87eb-82720ba4257f-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-6mt5b\" (UID: \"b18682b8-f91a-4c27-87eb-82720ba4257f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:47.327961 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.327934 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" Oct 30 18:37:47.328086 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.328039 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" Oct 30 18:37:47.328132 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.328115 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:47.330602 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.330590 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" Oct 30 18:37:47.338434 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.338415 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:47.341125 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.341107 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bscj4" Oct 30 18:37:47.782138 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.782101 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b"] Oct 30 18:37:47.787203 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.787184 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt"] Oct 30 18:37:47.824405 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:47.824193 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74153c90_cd72_43b8_85c5_aadb4d66a3e4.slice/crio-1e4fcc5170031f0801c53261863b1e6e0e08155878aa59fa803c67ed210bc1eb WatchSource:0}: Error finding container 1e4fcc5170031f0801c53261863b1e6e0e08155878aa59fa803c67ed210bc1eb: Status 404 returned error can't find the container with id 1e4fcc5170031f0801c53261863b1e6e0e08155878aa59fa803c67ed210bc1eb Oct 30 18:37:47.834425 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.834402 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:37:47.875258 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.875236 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bscj4"] Oct 30 18:37:47.879185 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.879153 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ltkzx"] Oct 30 18:37:47.879851 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.879834 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-dl96p"] Oct 30 18:37:47.884270 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:47.884241 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ccc9d41_f2e7_4b0d_a064_cb3e5549049a.slice/crio-8910ac2722f1327e268ab72c3da4a96e925c1f5314af6118b84b44292f31d6e0 WatchSource:0}: Error finding container 8910ac2722f1327e268ab72c3da4a96e925c1f5314af6118b84b44292f31d6e0: Status 404 returned error can't find the container with id 8910ac2722f1327e268ab72c3da4a96e925c1f5314af6118b84b44292f31d6e0 Oct 30 18:37:47.884396 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:47.884373 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6c975679c5-x5lg8"] Oct 30 18:37:47.884838 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:47.884817 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod447f2b64_f13e_48cc_94b9_e80df4a9d4f3.slice/crio-bd849bcedb31a82b08a6da91630fe52cbb0829fad0c10e8d9afdd20873d613da WatchSource:0}: Error finding container bd849bcedb31a82b08a6da91630fe52cbb0829fad0c10e8d9afdd20873d613da: Status 404 returned error can't find the container with id bd849bcedb31a82b08a6da91630fe52cbb0829fad0c10e8d9afdd20873d613da Oct 30 18:37:47.885808 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:47.885788 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8387e6ab_1e29_4ed8_9ee9_6b9595e4f5d7.slice/crio-cd745c637716976328327a22c804135220eaac4564ef3c373506d8eff38ad46e WatchSource:0}: Error finding container cd745c637716976328327a22c804135220eaac4564ef3c373506d8eff38ad46e: Status 404 returned error can't find the container with id cd745c637716976328327a22c804135220eaac4564ef3c373506d8eff38ad46e Oct 30 18:37:47.892381 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:37:47.892359 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1340fb2d_5f77_4316_a156_f82e0cb1124d.slice/crio-a2ccc42701ff2b0f5b7b80c3edb51ab6ce858cae108c59a9f4495b161c40575c WatchSource:0}: Error finding container a2ccc42701ff2b0f5b7b80c3edb51ab6ce858cae108c59a9f4495b161c40575c: Status 404 returned error can't find the container with id a2ccc42701ff2b0f5b7b80c3edb51ab6ce858cae108c59a9f4495b161c40575c Oct 30 18:37:48.207615 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:48.207584 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:37:48.524465 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:48.524370 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6c975679c5-x5lg8" event={"ID":"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7","Type":"ContainerStarted","Data":"ca8c3322f7d670587796d2d54fe32ed56e81fc36f7c04410d62a3bacf8832dc3"} Oct 30 18:37:48.524965 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:48.524938 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6c975679c5-x5lg8" event={"ID":"8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7","Type":"ContainerStarted","Data":"cd745c637716976328327a22c804135220eaac4564ef3c373506d8eff38ad46e"} Oct 30 18:37:48.525718 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:48.525691 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" event={"ID":"b18682b8-f91a-4c27-87eb-82720ba4257f","Type":"ContainerStarted","Data":"2fda68936955e1ef2ebca06bd2fa851e3613dd44980778dcd50c98e4986ca4b3"} Oct 30 18:37:48.527226 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:48.527205 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bscj4" event={"ID":"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a","Type":"ContainerStarted","Data":"2aa0dd9812aeaedbb132b4ecf77387ed1f3f478ac3b77fb245cb81ac592cc4fe"} Oct 30 18:37:48.527332 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:48.527253 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bscj4" event={"ID":"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a","Type":"ContainerStarted","Data":"8910ac2722f1327e268ab72c3da4a96e925c1f5314af6118b84b44292f31d6e0"} Oct 30 18:37:48.528484 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:48.528459 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ltkzx" event={"ID":"447f2b64-f13e-48cc-94b9-e80df4a9d4f3","Type":"ContainerStarted","Data":"bd849bcedb31a82b08a6da91630fe52cbb0829fad0c10e8d9afdd20873d613da"} Oct 30 18:37:48.529556 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:48.529531 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" event={"ID":"1340fb2d-5f77-4316-a156-f82e0cb1124d","Type":"ContainerStarted","Data":"a2ccc42701ff2b0f5b7b80c3edb51ab6ce858cae108c59a9f4495b161c40575c"} Oct 30 18:37:48.530625 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:48.530605 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" event={"ID":"74153c90-cd72-43b8-85c5-aadb4d66a3e4","Type":"ContainerStarted","Data":"1e4fcc5170031f0801c53261863b1e6e0e08155878aa59fa803c67ed210bc1eb"} Oct 30 18:37:48.727476 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:48.727445 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:48.727770 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:48.727745 5054 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:37:49.331116 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:49.329272 5054 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:49.333447 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:49.333426 5054 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:49.543450 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:49.543375 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bscj4" event={"ID":"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a","Type":"ContainerStarted","Data":"dad64053a9ac5539e884268260c65e66e49aa648cfa09854b98b82c2289628d4"} Oct 30 18:37:50.554831 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:50.554794 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-dl96p" event={"ID":"1340fb2d-5f77-4316-a156-f82e0cb1124d","Type":"ContainerStarted","Data":"48855d6fb020a3127d3ed98da83bb14f2762e28fd4021d26780eba22b5e5cbf5"} Oct 30 18:37:50.556866 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:50.556840 5054 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:37:50.556982 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:50.556889 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-cf8qt" event={"ID":"74153c90-cd72-43b8-85c5-aadb4d66a3e4","Type":"ContainerStarted","Data":"4329e221b7cf4737b4f48d708951e0ffa9cea4fc772c7b5ce06a71bf9fb754c0"} Oct 30 18:37:51.563177 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:51.563094 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bscj4" event={"ID":"3ccc9d41-f2e7-4b0d-a064-cb3e5549049a","Type":"ContainerStarted","Data":"ae1fa201642a6c353d72b4de2100a453a7dd713f9fb0fc9b63078b72d15b70e9"} Oct 30 18:37:51.564931 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:51.564900 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ltkzx" event={"ID":"447f2b64-f13e-48cc-94b9-e80df4a9d4f3","Type":"ContainerStarted","Data":"c992885e0f10ff76d85c12079ed9e9c7ed5a5a939f50c1fe4ab3bf12e5eebffc"} Oct 30 18:37:51.565081 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:51.564957 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ltkzx" event={"ID":"447f2b64-f13e-48cc-94b9-e80df4a9d4f3","Type":"ContainerStarted","Data":"6b425022a98a4c3c18999a33e37260f39cdd77f636621bf743860d4a9f1436f9"} Oct 30 18:37:51.565081 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:51.565030 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ltkzx" Oct 30 18:37:51.566572 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:51.566549 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" event={"ID":"b18682b8-f91a-4c27-87eb-82720ba4257f","Type":"ContainerStarted","Data":"9ac43bacdfe0fbab183e76a39f06be6ec9d2b9829f4a610d4a53be21369872a1"} Oct 30 18:37:51.566676 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:51.566596 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-6mt5b" event={"ID":"b18682b8-f91a-4c27-87eb-82720ba4257f","Type":"ContainerStarted","Data":"c6ce271e423b8a5f1fe15f21d1041f32b30eba62d1eb7408319c809601d65103"} Oct 30 18:37:51.960529 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:51.960498 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4hsc7" Oct 30 18:37:54.640823 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:54.640789 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:37:54.641263 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:54.640910 5054 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:37:54.642165 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:37:54.642144 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6c975679c5-x5lg8" Oct 30 18:38:00.098803 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.098771 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2dtth"] Oct 30 18:38:00.103546 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.103525 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2dtth" Oct 30 18:38:00.131945 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.131918 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-frnlr"] Oct 30 18:38:00.135206 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.135187 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-frnlr" Oct 30 18:38:00.147343 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.147319 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-649485556b-7x5gc"] Oct 30 18:38:00.151656 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.151629 5054 patch_prober.go:29] interesting pod/image-registry-649485556b-7x5gc container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Oct 30 18:38:00.151780 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.151704 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-649485556b-7x5gc" podUID="d9b465b9-97b6-47e2-8732-ed29b8f24ed0" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 18:38:00.153172 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.153149 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-67486b6598-97544"] Oct 30 18:38:00.156232 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.156213 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.178930 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.178903 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-2l7n4\"" Oct 30 18:38:00.185265 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.185239 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/049ceadf-4565-404c-a802-2b8c93770746-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-2dtth\" (UID: \"049ceadf-4565-404c-a802-2b8c93770746\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2dtth" Oct 30 18:38:00.193161 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.193141 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Oct 30 18:38:00.225226 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.225206 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Oct 30 18:38:00.225472 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.225454 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Oct 30 18:38:00.228590 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.228575 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Oct 30 18:38:00.229793 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.229772 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6bcc987885-s2p89"] Oct 30 18:38:00.233099 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.233083 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.240382 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.240362 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Oct 30 18:38:00.285784 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.285760 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/049ceadf-4565-404c-a802-2b8c93770746-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-2dtth\" (UID: \"049ceadf-4565-404c-a802-2b8c93770746\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2dtth" Oct 30 18:38:00.285889 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.285795 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvzx9\" (UniqueName: \"kubernetes.io/projected/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-kube-api-access-rvzx9\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.285889 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.285815 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g25bd\" (UniqueName: \"kubernetes.io/projected/89e68c92-6f5f-4be0-9cb1-c78298269a44-kube-api-access-g25bd\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.285889 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.285833 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-console-oauth-config\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.285889 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.285857 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-service-ca\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.286119 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.285906 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-console-config\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.286119 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.285935 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-oauth-serving-cert\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.286228 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.286113 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/89e68c92-6f5f-4be0-9cb1-c78298269a44-installation-pull-secrets\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.286228 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.286150 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6sjm\" (UniqueName: \"kubernetes.io/projected/4bd54db8-cc7f-4bee-9cb9-99da7a44807c-kube-api-access-l6sjm\") pod \"downloads-6dd789c8b7-frnlr\" (UID: \"4bd54db8-cc7f-4bee-9cb9-99da7a44807c\") " pod="openshift-console/downloads-6dd789c8b7-frnlr" Oct 30 18:38:00.286228 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.286176 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/89e68c92-6f5f-4be0-9cb1-c78298269a44-image-registry-private-configuration\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.286228 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.286200 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/89e68c92-6f5f-4be0-9cb1-c78298269a44-ca-trust-extracted\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.286392 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.286284 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/89e68c92-6f5f-4be0-9cb1-c78298269a44-trusted-ca\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.286392 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.286312 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bd54db8-cc7f-4bee-9cb9-99da7a44807c-tmp\") pod \"downloads-6dd789c8b7-frnlr\" (UID: \"4bd54db8-cc7f-4bee-9cb9-99da7a44807c\") " pod="openshift-console/downloads-6dd789c8b7-frnlr" Oct 30 18:38:00.286392 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.286329 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/89e68c92-6f5f-4be0-9cb1-c78298269a44-registry-tls\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.286392 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.286351 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-console-serving-cert\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.286548 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.286455 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-trusted-ca-bundle\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.286548 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.286485 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/89e68c92-6f5f-4be0-9cb1-c78298269a44-bound-sa-token\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.286548 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.286513 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/89e68c92-6f5f-4be0-9cb1-c78298269a44-registry-certificates\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.286656 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.286586 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-tmp\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.288853 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.288831 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Oct 30 18:38:00.289025 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.288981 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/049ceadf-4565-404c-a802-2b8c93770746-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-2dtth\" (UID: \"049ceadf-4565-404c-a802-2b8c93770746\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2dtth" Oct 30 18:38:00.289101 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.289034 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Oct 30 18:38:00.289141 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.289120 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Oct 30 18:38:00.295652 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.295636 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Oct 30 18:38:00.295652 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.295640 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-64lnc\"" Oct 30 18:38:00.320128 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.320105 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jqf2g\"" Oct 30 18:38:00.321273 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.321249 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-frnlr"] Oct 30 18:38:00.322075 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.322060 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67486b6598-97544"] Oct 30 18:38:00.322790 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.322770 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6bcc987885-s2p89"] Oct 30 18:38:00.386697 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.386670 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2dtth"] Oct 30 18:38:00.386963 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.386942 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-trusted-ca-bundle\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.387022 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.386978 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/89e68c92-6f5f-4be0-9cb1-c78298269a44-bound-sa-token\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.387067 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.387025 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/89e68c92-6f5f-4be0-9cb1-c78298269a44-registry-certificates\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.387113 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.387075 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-tmp\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.387152 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.387113 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvzx9\" (UniqueName: \"kubernetes.io/projected/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-kube-api-access-rvzx9\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.387152 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.387136 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g25bd\" (UniqueName: \"kubernetes.io/projected/89e68c92-6f5f-4be0-9cb1-c78298269a44-kube-api-access-g25bd\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.387238 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.387165 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-console-oauth-config\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.387238 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.387193 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-service-ca\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.387238 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.387219 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-console-config\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.387381 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.387240 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-oauth-serving-cert\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.387381 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.387280 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/89e68c92-6f5f-4be0-9cb1-c78298269a44-installation-pull-secrets\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.387381 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.387328 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6sjm\" (UniqueName: \"kubernetes.io/projected/4bd54db8-cc7f-4bee-9cb9-99da7a44807c-kube-api-access-l6sjm\") pod \"downloads-6dd789c8b7-frnlr\" (UID: \"4bd54db8-cc7f-4bee-9cb9-99da7a44807c\") " pod="openshift-console/downloads-6dd789c8b7-frnlr" Oct 30 18:38:00.387381 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.387368 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/89e68c92-6f5f-4be0-9cb1-c78298269a44-image-registry-private-configuration\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.387571 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.387398 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/89e68c92-6f5f-4be0-9cb1-c78298269a44-ca-trust-extracted\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.387571 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.387438 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/89e68c92-6f5f-4be0-9cb1-c78298269a44-trusted-ca\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.387571 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.387480 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bd54db8-cc7f-4bee-9cb9-99da7a44807c-tmp\") pod \"downloads-6dd789c8b7-frnlr\" (UID: \"4bd54db8-cc7f-4bee-9cb9-99da7a44807c\") " pod="openshift-console/downloads-6dd789c8b7-frnlr" Oct 30 18:38:00.387571 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.387511 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/89e68c92-6f5f-4be0-9cb1-c78298269a44-registry-tls\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.387571 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.387546 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-tmp\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.387980 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.387953 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-oauth-serving-cert\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.388221 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.387547 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-console-serving-cert\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.388334 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.388231 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-service-ca\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.388334 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.388319 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4bd54db8-cc7f-4bee-9cb9-99da7a44807c-tmp\") pod \"downloads-6dd789c8b7-frnlr\" (UID: \"4bd54db8-cc7f-4bee-9cb9-99da7a44807c\") " pod="openshift-console/downloads-6dd789c8b7-frnlr" Oct 30 18:38:00.388514 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.388491 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-trusted-ca-bundle\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.389096 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.389053 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-console-config\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.389676 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.389648 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/89e68c92-6f5f-4be0-9cb1-c78298269a44-trusted-ca\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.389843 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.389821 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/89e68c92-6f5f-4be0-9cb1-c78298269a44-ca-trust-extracted\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.390574 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.390550 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/89e68c92-6f5f-4be0-9cb1-c78298269a44-registry-certificates\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.391401 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.391381 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/89e68c92-6f5f-4be0-9cb1-c78298269a44-registry-tls\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.391498 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.391457 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-console-oauth-config\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.391747 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.391722 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/89e68c92-6f5f-4be0-9cb1-c78298269a44-installation-pull-secrets\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.391879 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.391860 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/89e68c92-6f5f-4be0-9cb1-c78298269a44-image-registry-private-configuration\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.392232 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.392210 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-console-serving-cert\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.414943 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.414920 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2dtth" Oct 30 18:38:00.464215 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.464191 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvzx9\" (UniqueName: \"kubernetes.io/projected/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-kube-api-access-rvzx9\") pod \"console-67486b6598-97544\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.465078 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.465050 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g25bd\" (UniqueName: \"kubernetes.io/projected/89e68c92-6f5f-4be0-9cb1-c78298269a44-kube-api-access-g25bd\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.467585 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.467563 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/89e68c92-6f5f-4be0-9cb1-c78298269a44-bound-sa-token\") pod \"image-registry-6bcc987885-s2p89\" (UID: \"89e68c92-6f5f-4be0-9cb1-c78298269a44\") " pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.469916 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.469210 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6sjm\" (UniqueName: \"kubernetes.io/projected/4bd54db8-cc7f-4bee-9cb9-99da7a44807c-kube-api-access-l6sjm\") pod \"downloads-6dd789c8b7-frnlr\" (UID: \"4bd54db8-cc7f-4bee-9cb9-99da7a44807c\") " pod="openshift-console/downloads-6dd789c8b7-frnlr" Oct 30 18:38:00.472681 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.472629 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:00.544655 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.544623 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:00.747210 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.747142 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-frnlr" Oct 30 18:38:00.795059 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.794442 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2dtth"] Oct 30 18:38:00.806522 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:38:00.805586 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod049ceadf_4565_404c_a802_2b8c93770746.slice/crio-39b5d80d110b790c23deaf5de90bc7fd723af1056e2425f27e513da84c028dc0 WatchSource:0}: Error finding container 39b5d80d110b790c23deaf5de90bc7fd723af1056e2425f27e513da84c028dc0: Status 404 returned error can't find the container with id 39b5d80d110b790c23deaf5de90bc7fd723af1056e2425f27e513da84c028dc0 Oct 30 18:38:00.875933 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:00.875900 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67486b6598-97544"] Oct 30 18:38:00.882068 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:38:00.882038 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb83b2ad9_2bf1_4e61_b0b8_5c72761bd34c.slice/crio-bb84c7e691d1a2e0737148c677bda973e56154351b6af3f2a31852086ebe28b4 WatchSource:0}: Error finding container bb84c7e691d1a2e0737148c677bda973e56154351b6af3f2a31852086ebe28b4: Status 404 returned error can't find the container with id bb84c7e691d1a2e0737148c677bda973e56154351b6af3f2a31852086ebe28b4 Oct 30 18:38:01.068955 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:01.068879 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6bcc987885-s2p89"] Oct 30 18:38:01.074656 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:38:01.074627 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89e68c92_6f5f_4be0_9cb1_c78298269a44.slice/crio-cd56d2556cd87809581bdb242bedfb8a50250b177e5605f7d81029f51f371c50 WatchSource:0}: Error finding container cd56d2556cd87809581bdb242bedfb8a50250b177e5605f7d81029f51f371c50: Status 404 returned error can't find the container with id cd56d2556cd87809581bdb242bedfb8a50250b177e5605f7d81029f51f371c50 Oct 30 18:38:01.131294 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:01.131266 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-frnlr"] Oct 30 18:38:01.136761 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:38:01.136739 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bd54db8_cc7f_4bee_9cb9_99da7a44807c.slice/crio-4ef44070cd71be1deb92cc7f813a14752331a267367888a04734bf631a4755ce WatchSource:0}: Error finding container 4ef44070cd71be1deb92cc7f813a14752331a267367888a04734bf631a4755ce: Status 404 returned error can't find the container with id 4ef44070cd71be1deb92cc7f813a14752331a267367888a04734bf631a4755ce Oct 30 18:38:01.573145 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:01.573116 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ltkzx" Oct 30 18:38:01.609896 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:01.609863 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bcc987885-s2p89" event={"ID":"89e68c92-6f5f-4be0-9cb1-c78298269a44","Type":"ContainerStarted","Data":"1b0d6264e9c28e8315f6eb7f8f1d7e25876b9e312cfcb1ad1e29e65e829f1004"} Oct 30 18:38:01.610051 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:01.609912 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bcc987885-s2p89" event={"ID":"89e68c92-6f5f-4be0-9cb1-c78298269a44","Type":"ContainerStarted","Data":"cd56d2556cd87809581bdb242bedfb8a50250b177e5605f7d81029f51f371c50"} Oct 30 18:38:01.610051 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:01.609939 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:01.611049 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:01.611027 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2dtth" event={"ID":"049ceadf-4565-404c-a802-2b8c93770746","Type":"ContainerStarted","Data":"39b5d80d110b790c23deaf5de90bc7fd723af1056e2425f27e513da84c028dc0"} Oct 30 18:38:01.612104 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:01.612083 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-frnlr" event={"ID":"4bd54db8-cc7f-4bee-9cb9-99da7a44807c","Type":"ContainerStarted","Data":"4ef44070cd71be1deb92cc7f813a14752331a267367888a04734bf631a4755ce"} Oct 30 18:38:01.613105 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:01.613086 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67486b6598-97544" event={"ID":"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c","Type":"ContainerStarted","Data":"bb84c7e691d1a2e0737148c677bda973e56154351b6af3f2a31852086ebe28b4"} Oct 30 18:38:01.817532 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:01.817471 5054 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6bcc987885-s2p89" podStartSLOduration=1.817455465 podStartE2EDuration="1.817455465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:01.816286787 +0000 UTC m=+23.544665505" watchObservedRunningTime="2026-10-30 18:38:01.817455465 +0000 UTC m=+23.545834205" Oct 30 18:38:02.429552 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:02.429512 5054 patch_prober.go:29] interesting pod/image-registry-5df4b54dc9-8pdtm container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Oct 30 18:38:02.430029 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:02.429583 5054 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" podUID="7fcdea8f-3b0f-42c5-bb3d-b2266b046491" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 18:38:02.622785 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:02.621253 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2dtth" event={"ID":"049ceadf-4565-404c-a802-2b8c93770746","Type":"ContainerStarted","Data":"e1a62bb7f3fb73a7a0411470c51eb581411775b3c336700323a185c5ba9bb86f"} Oct 30 18:38:02.622785 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:02.621530 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2dtth" Oct 30 18:38:02.627185 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:02.627158 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2dtth" Oct 30 18:38:02.740423 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:02.740357 5054 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-2dtth" podStartSLOduration=1.340413067 podStartE2EDuration="2.740342267s" totalImagesPullingTime="1.3999292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:02.739520446 +0000 UTC m=+24.467899164" watchObservedRunningTime="2026-10-30 18:38:02.740342267 +0000 UTC m=+24.468720986" Oct 30 18:38:05.637342 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:05.637298 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67486b6598-97544" event={"ID":"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c","Type":"ContainerStarted","Data":"7a77cdd9bc411b51855df0112820fa330baa68908887b8cd0e54b2cab460a794"} Oct 30 18:38:05.755377 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:05.755322 5054 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-67486b6598-97544" podStartSLOduration=1.362054309 podStartE2EDuration="5.75530213s" totalImagesPullingTime="4.393247821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:05.754303715 +0000 UTC m=+27.482682431" watchObservedRunningTime="2026-10-30 18:38:05.75530213 +0000 UTC m=+27.483680850" Oct 30 18:38:06.515454 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:06.515423 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:38:08.211688 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:08.211654 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-62vfn" Oct 30 18:38:08.601790 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:38:08.601711 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:38:10.153209 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.153176 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:38:10.472926 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.472840 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:10.472926 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.472905 5054 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:10.478794 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.478768 5054 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:10.595033 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.594988 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mmrxr"] Oct 30 18:38:10.630181 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.630153 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.682923 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.682893 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:10.733149 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.733087 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 30 18:38:10.733149 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.733095 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 30 18:38:10.733302 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.733156 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-swgzb\"" Oct 30 18:38:10.789969 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.789938 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.790154 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.790132 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-node-exporter-textfile\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.790223 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.790191 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-node-exporter-wtmp\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.790223 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.790213 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78xmh\" (UniqueName: \"kubernetes.io/projected/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-kube-api-access-78xmh\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.790324 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.790251 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-node-exporter-tls\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.790324 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.790269 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-metrics-client-ca\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.790430 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.790332 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-root\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.790430 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.790348 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-node-exporter-accelerators-collector-config\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.790430 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.790366 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-sys\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.791660 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.791612 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 30 18:38:10.791660 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.791648 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 30 18:38:10.891733 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.891691 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.891906 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.891769 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-node-exporter-textfile\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.891906 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.891798 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-node-exporter-wtmp\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.891906 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.891825 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78xmh\" (UniqueName: \"kubernetes.io/projected/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-kube-api-access-78xmh\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.891906 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.891881 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-node-exporter-tls\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.891906 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.891903 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-metrics-client-ca\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.892220 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.891959 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-root\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.892220 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.891971 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-node-exporter-wtmp\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.892220 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.891984 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-node-exporter-accelerators-collector-config\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.892220 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.892182 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-root\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.892484 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.892230 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-node-exporter-textfile\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.892484 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.892237 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-sys\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.892484 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.892279 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-sys\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.892783 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.892759 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-metrics-client-ca\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.892856 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.892787 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-node-exporter-accelerators-collector-config\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.894305 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.894281 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.894550 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.894522 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-node-exporter-tls\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.960383 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.960321 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78xmh\" (UniqueName: \"kubernetes.io/projected/c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2-kube-api-access-78xmh\") pod \"node-exporter-mmrxr\" (UID: \"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2\") " pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:10.998246 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:10.998180 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mmrxr" Oct 30 18:38:11.043091 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:38:11.042996 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc70d1b0d_1cd4_40c2_8ee9_08f2a07315e2.slice/crio-678f53242f71e4ebd472a8f37f532bb75608ad9fa44d53afb2a0879f3e8ed5b6 WatchSource:0}: Error finding container 678f53242f71e4ebd472a8f37f532bb75608ad9fa44d53afb2a0879f3e8ed5b6: Status 404 returned error can't find the container with id 678f53242f71e4ebd472a8f37f532bb75608ad9fa44d53afb2a0879f3e8ed5b6 Oct 30 18:38:11.665316 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:11.665265 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mmrxr" event={"ID":"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2","Type":"ContainerStarted","Data":"678f53242f71e4ebd472a8f37f532bb75608ad9fa44d53afb2a0879f3e8ed5b6"} Oct 30 18:38:13.676497 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:13.676359 5054 generic.go:356] "Generic (PLEG): container finished" podID="c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2" containerID="626b98528fe25696fb70980a5aa2ad0c6692d1c5b2a139657cc171841ef238d9" exitCode=0 Oct 30 18:38:13.676497 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:13.676450 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mmrxr" event={"ID":"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2","Type":"ContainerDied","Data":"626b98528fe25696fb70980a5aa2ad0c6692d1c5b2a139657cc171841ef238d9"} Oct 30 18:38:14.684019 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:14.683976 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mmrxr" event={"ID":"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2","Type":"ContainerStarted","Data":"17a9e806d3c488e91bb183c026e6db5b8e313baa0f033265dc19d32b050a0eb9"} Oct 30 18:38:14.684487 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:14.684060 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mmrxr" event={"ID":"c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2","Type":"ContainerStarted","Data":"9b19a61b5ec1d79f97b1acb53b81240924c68dc06c37e605a4726299c5f7072e"} Oct 30 18:38:14.891083 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:14.890997 5054 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mmrxr" podStartSLOduration=2.841155433 podStartE2EDuration="4.890982507s" totalImagesPullingTime="1.959421298s" totalInitContainerRuntime="90.405776ms" isStatefulPod=true podCreationTimestamp="2026-10-30 18:38:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:14.86147497 +0000 UTC m=+36.589853687" watchObservedRunningTime="2026-10-30 18:38:14.890982507 +0000 UTC m=+36.619361222" Oct 30 18:38:18.188541 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.188505 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-568f944-hrvh5"] Oct 30 18:38:18.191913 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.191888 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.258422 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.258387 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-console-config\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.258565 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.258445 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-oauth-serving-cert\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.258639 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.258558 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-console-oauth-config\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.258639 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.258629 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24lrn\" (UniqueName: \"kubernetes.io/projected/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-kube-api-access-24lrn\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.258714 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.258658 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-service-ca\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.258747 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.258715 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-console-serving-cert\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.258789 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.258737 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-568f944-hrvh5"] Oct 30 18:38:18.258789 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.258746 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-trusted-ca-bundle\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.258862 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.258824 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-tmp\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.359562 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.359516 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-trusted-ca-bundle\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.359751 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.359669 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-tmp\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.359751 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.359722 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-console-config\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.359875 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.359755 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-oauth-serving-cert\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.359875 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.359785 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-console-oauth-config\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.359875 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.359809 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24lrn\" (UniqueName: \"kubernetes.io/projected/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-kube-api-access-24lrn\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.359875 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.359851 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-service-ca\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.360121 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.359920 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-console-serving-cert\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.360121 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.360082 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-tmp\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.360445 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.360416 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-trusted-ca-bundle\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.360972 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.360946 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-console-config\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.360972 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.360963 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-oauth-serving-cert\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.361213 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.361191 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-service-ca\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.362751 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.362731 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-console-oauth-config\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.362913 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.362892 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-console-serving-cert\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.460943 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.460862 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24lrn\" (UniqueName: \"kubernetes.io/projected/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-kube-api-access-24lrn\") pod \"console-568f944-hrvh5\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.505591 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.505558 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:18.931872 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:18.931837 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-568f944-hrvh5"] Oct 30 18:38:18.937882 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:38:18.937853 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda63d77ef_dc01_4e7d_bb48_7657f2e7d626.slice/crio-b82fd16a8e9ab6b4302e8b8ebaebbbde35bf07ac545ae7a16a5b1fb56294533a WatchSource:0}: Error finding container b82fd16a8e9ab6b4302e8b8ebaebbbde35bf07ac545ae7a16a5b1fb56294533a: Status 404 returned error can't find the container with id b82fd16a8e9ab6b4302e8b8ebaebbbde35bf07ac545ae7a16a5b1fb56294533a Oct 30 18:38:19.710129 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:19.710081 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-568f944-hrvh5" event={"ID":"a63d77ef-dc01-4e7d-bb48-7657f2e7d626","Type":"ContainerStarted","Data":"f8ebf6873deefd161e1e09ae2167898eaa96dbcc11b6dfc0eb8f509d411f0f56"} Oct 30 18:38:19.710734 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:19.710147 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-568f944-hrvh5" event={"ID":"a63d77ef-dc01-4e7d-bb48-7657f2e7d626","Type":"ContainerStarted","Data":"b82fd16a8e9ab6b4302e8b8ebaebbbde35bf07ac545ae7a16a5b1fb56294533a"} Oct 30 18:38:19.712096 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:19.712062 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-frnlr" event={"ID":"4bd54db8-cc7f-4bee-9cb9-99da7a44807c","Type":"ContainerStarted","Data":"0cbc8e4940be5b1a91b81cbef60ee8c05b9d98d7157ebfbf436ae7d70dc35177"} Oct 30 18:38:19.712424 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:19.712406 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-frnlr" Oct 30 18:38:19.729971 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:19.729946 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-frnlr" Oct 30 18:38:19.838477 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:19.838422 5054 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-568f944-hrvh5" podStartSLOduration=1.838400737 podStartE2EDuration="1.838400737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:19.838402282 +0000 UTC m=+41.566780999" watchObservedRunningTime="2026-10-30 18:38:19.838400737 +0000 UTC m=+41.566779454" Oct 30 18:38:19.968292 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:19.968186 5054 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-frnlr" podStartSLOduration=1.99969694 podStartE2EDuration="19.968167057s" totalImagesPullingTime="17.968470117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:19.965861595 +0000 UTC m=+41.694240311" watchObservedRunningTime="2026-10-30 18:38:19.968167057 +0000 UTC m=+41.696545779" Oct 30 18:38:22.626827 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:22.626794 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6bcc987885-s2p89" Oct 30 18:38:22.886081 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:22.885984 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5df4b54dc9-8pdtm"] Oct 30 18:38:25.169170 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.169119 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-649485556b-7x5gc" podUID="d9b465b9-97b6-47e2-8732-ed29b8f24ed0" containerName="registry" containerID="cri-o://5ac2d38334da6fc2906b90398de622fbd49104f8b8a3ea6afbf3c8dfb2d22f48" gracePeriod=30 Oct 30 18:38:25.473341 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.473309 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:38:25.529876 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.529842 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-bound-sa-token\") pod \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " Oct 30 18:38:25.530054 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.529913 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-ca-trust-extracted\") pod \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " Oct 30 18:38:25.530139 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.530061 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-certificates\") pod \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " Oct 30 18:38:25.530139 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.530121 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-image-registry-private-configuration\") pod \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " Oct 30 18:38:25.530256 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.530183 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-trusted-ca\") pod \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " Oct 30 18:38:25.530256 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.530233 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-installation-pull-secrets\") pod \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " Oct 30 18:38:25.530357 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.530275 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-kube-api-access-dw7kf\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-kube-api-access-dw7kf\") pod \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " Oct 30 18:38:25.530357 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.530347 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls\") pod \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\" (UID: \"d9b465b9-97b6-47e2-8732-ed29b8f24ed0\") " Oct 30 18:38:25.530463 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.530366 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-certificates" pod "d9b465b9-97b6-47e2-8732-ed29b8f24ed0" (UID: "d9b465b9-97b6-47e2-8732-ed29b8f24ed0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:38:25.530930 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.530628 5054 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-certificates\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:25.530930 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.530709 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-trusted-ca" pod "d9b465b9-97b6-47e2-8732-ed29b8f24ed0" (UID: "d9b465b9-97b6-47e2-8732-ed29b8f24ed0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:38:25.533718 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.533687 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-image-registry-private-configuration" pod "d9b465b9-97b6-47e2-8732-ed29b8f24ed0" (UID: "d9b465b9-97b6-47e2-8732-ed29b8f24ed0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:25.534644 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.534616 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-bound-sa-token" pod "d9b465b9-97b6-47e2-8732-ed29b8f24ed0" (UID: "d9b465b9-97b6-47e2-8732-ed29b8f24ed0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:38:25.534748 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.534640 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls" pod "d9b465b9-97b6-47e2-8732-ed29b8f24ed0" (UID: "d9b465b9-97b6-47e2-8732-ed29b8f24ed0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:38:25.534809 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.534776 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-kube-api-access-dw7kf" pod "d9b465b9-97b6-47e2-8732-ed29b8f24ed0" (UID: "d9b465b9-97b6-47e2-8732-ed29b8f24ed0"). InnerVolumeSpecName "kube-api-access-dw7kf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:38:25.535070 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.535043 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-installation-pull-secrets" pod "d9b465b9-97b6-47e2-8732-ed29b8f24ed0" (UID: "d9b465b9-97b6-47e2-8732-ed29b8f24ed0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:25.538833 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.538810 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-ca-trust-extracted" pod "d9b465b9-97b6-47e2-8732-ed29b8f24ed0" (UID: "d9b465b9-97b6-47e2-8732-ed29b8f24ed0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:38:25.631649 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.631614 5054 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-registry-tls\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:25.631649 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.631643 5054 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-bound-sa-token\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:25.631649 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.631654 5054 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-ca-trust-extracted\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:25.631860 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.631663 5054 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-image-registry-private-configuration\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:25.631860 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.631673 5054 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-trusted-ca\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:25.631860 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.631681 5054 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-installation-pull-secrets\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:25.631860 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.631691 5054 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dw7kf\" (UniqueName: \"kubernetes.io/projected/d9b465b9-97b6-47e2-8732-ed29b8f24ed0-kube-api-access-dw7kf\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:25.743058 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.742945 5054 generic.go:356] "Generic (PLEG): container finished" podID="d9b465b9-97b6-47e2-8732-ed29b8f24ed0" containerID="5ac2d38334da6fc2906b90398de622fbd49104f8b8a3ea6afbf3c8dfb2d22f48" exitCode=0 Oct 30 18:38:25.743058 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.743030 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-649485556b-7x5gc" Oct 30 18:38:25.743058 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.743035 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-649485556b-7x5gc" event={"ID":"d9b465b9-97b6-47e2-8732-ed29b8f24ed0","Type":"ContainerDied","Data":"5ac2d38334da6fc2906b90398de622fbd49104f8b8a3ea6afbf3c8dfb2d22f48"} Oct 30 18:38:25.743378 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.743095 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-649485556b-7x5gc" event={"ID":"d9b465b9-97b6-47e2-8732-ed29b8f24ed0","Type":"ContainerDied","Data":"e33b01a9db9bbb98f5ad302eaf29eac84dc00c5c47636d085bb9a314fe877f5a"} Oct 30 18:38:25.743378 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.743127 5054 scope.go:122] "RemoveContainer" containerID="5ac2d38334da6fc2906b90398de622fbd49104f8b8a3ea6afbf3c8dfb2d22f48" Oct 30 18:38:25.772413 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.772386 5054 scope.go:122] "RemoveContainer" containerID="5ac2d38334da6fc2906b90398de622fbd49104f8b8a3ea6afbf3c8dfb2d22f48" Oct 30 18:38:25.772763 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:38:25.772739 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ac2d38334da6fc2906b90398de622fbd49104f8b8a3ea6afbf3c8dfb2d22f48\": container with ID starting with 5ac2d38334da6fc2906b90398de622fbd49104f8b8a3ea6afbf3c8dfb2d22f48 not found: ID does not exist" containerID="5ac2d38334da6fc2906b90398de622fbd49104f8b8a3ea6afbf3c8dfb2d22f48" Oct 30 18:38:25.772860 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.772774 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ac2d38334da6fc2906b90398de622fbd49104f8b8a3ea6afbf3c8dfb2d22f48"} err="failed to get container status \"5ac2d38334da6fc2906b90398de622fbd49104f8b8a3ea6afbf3c8dfb2d22f48\": rpc error: code = NotFound desc = could not find container \"5ac2d38334da6fc2906b90398de622fbd49104f8b8a3ea6afbf3c8dfb2d22f48\": container with ID starting with 5ac2d38334da6fc2906b90398de622fbd49104f8b8a3ea6afbf3c8dfb2d22f48 not found: ID does not exist" Oct 30 18:38:25.897165 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.897130 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-649485556b-7x5gc"] Oct 30 18:38:25.910279 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:25.910244 5054 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-649485556b-7x5gc"] Oct 30 18:38:26.423238 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:26.423199 5054 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9b465b9-97b6-47e2-8732-ed29b8f24ed0" path="/var/lib/kubelet/pods/d9b465b9-97b6-47e2-8732-ed29b8f24ed0/volumes" Oct 30 18:38:28.505837 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:28.505802 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:28.506332 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:28.505863 5054 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:28.510889 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:28.510867 5054 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:28.760962 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:28.760891 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-568f944-hrvh5" Oct 30 18:38:28.947201 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:28.947164 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67486b6598-97544"] Oct 30 18:38:38.596540 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:38:38.596499 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:38:47.908646 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:47.908584 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" podUID="7fcdea8f-3b0f-42c5-bb3d-b2266b046491" containerName="registry" containerID="cri-o://31426b35dccce5ef8e1411971efb6b5ab9c9b15ddc6fdc8bc01ed75a905773af" gracePeriod=30 Oct 30 18:38:48.224990 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.224965 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:38:48.336462 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.336429 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-ca-trust-extracted\") pod \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " Oct 30 18:38:48.336643 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.336472 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-kube-api-access-t24cp\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-kube-api-access-t24cp\") pod \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " Oct 30 18:38:48.336643 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.336496 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-image-registry-private-configuration\") pod \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " Oct 30 18:38:48.336643 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.336527 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-bound-sa-token\") pod \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " Oct 30 18:38:48.336643 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.336554 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls\") pod \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " Oct 30 18:38:48.336643 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.336593 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-installation-pull-secrets\") pod \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " Oct 30 18:38:48.336921 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.336668 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-trusted-ca\") pod \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " Oct 30 18:38:48.336921 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.336704 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-certificates\") pod \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\" (UID: \"7fcdea8f-3b0f-42c5-bb3d-b2266b046491\") " Oct 30 18:38:48.337404 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.337372 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-certificates" pod "7fcdea8f-3b0f-42c5-bb3d-b2266b046491" (UID: "7fcdea8f-3b0f-42c5-bb3d-b2266b046491"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:38:48.337514 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.337431 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-trusted-ca" pod "7fcdea8f-3b0f-42c5-bb3d-b2266b046491" (UID: "7fcdea8f-3b0f-42c5-bb3d-b2266b046491"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:38:48.339379 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.339260 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-image-registry-private-configuration" pod "7fcdea8f-3b0f-42c5-bb3d-b2266b046491" (UID: "7fcdea8f-3b0f-42c5-bb3d-b2266b046491"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:48.339379 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.339324 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-bound-sa-token" pod "7fcdea8f-3b0f-42c5-bb3d-b2266b046491" (UID: "7fcdea8f-3b0f-42c5-bb3d-b2266b046491"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:38:48.339516 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.339429 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls" pod "7fcdea8f-3b0f-42c5-bb3d-b2266b046491" (UID: "7fcdea8f-3b0f-42c5-bb3d-b2266b046491"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:38:48.339516 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.339442 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-kube-api-access-t24cp" pod "7fcdea8f-3b0f-42c5-bb3d-b2266b046491" (UID: "7fcdea8f-3b0f-42c5-bb3d-b2266b046491"). InnerVolumeSpecName "kube-api-access-t24cp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:38:48.339582 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.339518 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-installation-pull-secrets" pod "7fcdea8f-3b0f-42c5-bb3d-b2266b046491" (UID: "7fcdea8f-3b0f-42c5-bb3d-b2266b046491"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:48.345525 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.345498 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-ca-trust-extracted" pod "7fcdea8f-3b0f-42c5-bb3d-b2266b046491" (UID: "7fcdea8f-3b0f-42c5-bb3d-b2266b046491"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:38:48.437792 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.437767 5054 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-installation-pull-secrets\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:48.437792 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.437791 5054 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-trusted-ca\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:48.437932 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.437802 5054 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-certificates\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:48.437932 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.437811 5054 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-ca-trust-extracted\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:48.437932 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.437821 5054 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t24cp\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-kube-api-access-t24cp\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:48.437932 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.437832 5054 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-image-registry-private-configuration\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:48.437932 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.437840 5054 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-bound-sa-token\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:48.437932 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.437849 5054 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7fcdea8f-3b0f-42c5-bb3d-b2266b046491-registry-tls\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:48.842348 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.842262 5054 generic.go:356] "Generic (PLEG): container finished" podID="13bb697e-8bff-4a30-9e1d-b9f3baac416e" containerID="ab324550baf6d4c95d87cb945e2000659647cc9a310fb6886a8103b790ebd1b7" exitCode=0 Oct 30 18:38:48.842513 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.842339 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq" event={"ID":"13bb697e-8bff-4a30-9e1d-b9f3baac416e","Type":"ContainerDied","Data":"ab324550baf6d4c95d87cb945e2000659647cc9a310fb6886a8103b790ebd1b7"} Oct 30 18:38:48.842718 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.842700 5054 scope.go:122] "RemoveContainer" containerID="ab324550baf6d4c95d87cb945e2000659647cc9a310fb6886a8103b790ebd1b7" Oct 30 18:38:48.843651 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.843627 5054 generic.go:356] "Generic (PLEG): container finished" podID="7fcdea8f-3b0f-42c5-bb3d-b2266b046491" containerID="31426b35dccce5ef8e1411971efb6b5ab9c9b15ddc6fdc8bc01ed75a905773af" exitCode=0 Oct 30 18:38:48.843764 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.843652 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" event={"ID":"7fcdea8f-3b0f-42c5-bb3d-b2266b046491","Type":"ContainerDied","Data":"31426b35dccce5ef8e1411971efb6b5ab9c9b15ddc6fdc8bc01ed75a905773af"} Oct 30 18:38:48.843764 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.843698 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" event={"ID":"7fcdea8f-3b0f-42c5-bb3d-b2266b046491","Type":"ContainerDied","Data":"3dbf5737a4e81ce7dffe8835c803744bc14134b845b162a0485f14815e3b6f0f"} Oct 30 18:38:48.843764 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.843712 5054 scope.go:122] "RemoveContainer" containerID="31426b35dccce5ef8e1411971efb6b5ab9c9b15ddc6fdc8bc01ed75a905773af" Oct 30 18:38:48.843764 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.843698 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5df4b54dc9-8pdtm" Oct 30 18:38:48.845175 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.845151 5054 generic.go:356] "Generic (PLEG): container finished" podID="1704ffe1-d69c-4dd2-b7f6-5880bf1551f7" containerID="59b4e08b3189ba3020d2900961e1de77a72af71cf7ebc273a2938270917767bd" exitCode=0 Oct 30 18:38:48.845268 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.845189 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6" event={"ID":"1704ffe1-d69c-4dd2-b7f6-5880bf1551f7","Type":"ContainerDied","Data":"59b4e08b3189ba3020d2900961e1de77a72af71cf7ebc273a2938270917767bd"} Oct 30 18:38:48.845439 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.845423 5054 scope.go:122] "RemoveContainer" containerID="59b4e08b3189ba3020d2900961e1de77a72af71cf7ebc273a2938270917767bd" Oct 30 18:38:48.855201 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.855187 5054 scope.go:122] "RemoveContainer" containerID="31426b35dccce5ef8e1411971efb6b5ab9c9b15ddc6fdc8bc01ed75a905773af" Oct 30 18:38:48.855554 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:38:48.855536 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31426b35dccce5ef8e1411971efb6b5ab9c9b15ddc6fdc8bc01ed75a905773af\": container with ID starting with 31426b35dccce5ef8e1411971efb6b5ab9c9b15ddc6fdc8bc01ed75a905773af not found: ID does not exist" containerID="31426b35dccce5ef8e1411971efb6b5ab9c9b15ddc6fdc8bc01ed75a905773af" Oct 30 18:38:48.855603 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:48.855562 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31426b35dccce5ef8e1411971efb6b5ab9c9b15ddc6fdc8bc01ed75a905773af"} err="failed to get container status \"31426b35dccce5ef8e1411971efb6b5ab9c9b15ddc6fdc8bc01ed75a905773af\": rpc error: code = NotFound desc = could not find container \"31426b35dccce5ef8e1411971efb6b5ab9c9b15ddc6fdc8bc01ed75a905773af\": container with ID starting with 31426b35dccce5ef8e1411971efb6b5ab9c9b15ddc6fdc8bc01ed75a905773af not found: ID does not exist" Oct 30 18:38:49.252583 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:49.252545 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5df4b54dc9-8pdtm"] Oct 30 18:38:49.265778 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:49.265754 5054 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5df4b54dc9-8pdtm"] Oct 30 18:38:49.851441 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:49.851408 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-gktbq" event={"ID":"13bb697e-8bff-4a30-9e1d-b9f3baac416e","Type":"ContainerStarted","Data":"5a1d0b9e32bfcf002cccad113b66bb635192e0c856533c7c1692e7ee66a8cfb4"} Oct 30 18:38:49.854188 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:49.854163 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b2rv6" event={"ID":"1704ffe1-d69c-4dd2-b7f6-5880bf1551f7","Type":"ContainerStarted","Data":"a3228dd846e525d77126e8fe8d43047a0bbac761b010e23d19be39745c3cb66a"} Oct 30 18:38:50.421585 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:50.421552 5054 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7fcdea8f-3b0f-42c5-bb3d-b2266b046491" path="/var/lib/kubelet/pods/7fcdea8f-3b0f-42c5-bb3d-b2266b046491/volumes" Oct 30 18:38:53.967356 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:53.967319 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-67486b6598-97544" podUID="b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c" containerName="console" containerID="cri-o://7a77cdd9bc411b51855df0112820fa330baa68908887b8cd0e54b2cab460a794" gracePeriod=15 Oct 30 18:38:54.262564 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.262535 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:54.391433 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.391403 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-console-serving-cert\") pod \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " Oct 30 18:38:54.391598 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.391450 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-kube-api-access-rvzx9\" (UniqueName: \"kubernetes.io/projected/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-kube-api-access-rvzx9\") pod \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " Oct 30 18:38:54.391598 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.391489 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-service-ca\" (UniqueName: \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-service-ca\") pod \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " Oct 30 18:38:54.391598 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.391521 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-oauth-serving-cert\") pod \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " Oct 30 18:38:54.391598 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.391560 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-console-config\" (UniqueName: \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-console-config\") pod \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " Oct 30 18:38:54.391805 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.391610 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-trusted-ca-bundle\") pod \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " Oct 30 18:38:54.391805 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.391638 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-tmp\" (UniqueName: \"kubernetes.io/empty-dir/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-tmp\") pod \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " Oct 30 18:38:54.391805 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.391690 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-console-oauth-config\") pod \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\" (UID: \"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c\") " Oct 30 18:38:54.392272 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.392053 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-service-ca" pod "b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c" (UID: "b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:38:54.392272 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.392107 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-tmp" pod "b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c" (UID: "b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:38:54.392272 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.392162 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-trusted-ca-bundle" pod "b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c" (UID: "b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:38:54.392272 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.392205 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-oauth-serving-cert" pod "b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c" (UID: "b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:38:54.392272 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.392252 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-console-config" pod "b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c" (UID: "b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:38:54.394052 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.393969 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-kube-api-access-rvzx9" pod "b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c" (UID: "b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c"). InnerVolumeSpecName "kube-api-access-rvzx9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:38:54.394188 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.394163 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-console-serving-cert" pod "b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c" (UID: "b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:54.394264 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.394224 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-console-oauth-config" pod "b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c" (UID: "b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:54.492678 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.492618 5054 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-console-serving-cert\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:54.492678 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.492643 5054 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvzx9\" (UniqueName: \"kubernetes.io/projected/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-kube-api-access-rvzx9\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:54.492678 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.492654 5054 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-service-ca\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:54.492678 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.492663 5054 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-oauth-serving-cert\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:54.492678 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.492671 5054 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-console-config\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:54.492678 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.492679 5054 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-trusted-ca-bundle\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:54.492932 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.492687 5054 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-tmp\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:54.492932 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.492697 5054 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c-console-oauth-config\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:38:54.874320 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.874282 5054 generic.go:356] "Generic (PLEG): container finished" podID="b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c" containerID="7a77cdd9bc411b51855df0112820fa330baa68908887b8cd0e54b2cab460a794" exitCode=0 Oct 30 18:38:54.874473 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.874348 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67486b6598-97544" Oct 30 18:38:54.874473 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.874352 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67486b6598-97544" event={"ID":"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c","Type":"ContainerDied","Data":"7a77cdd9bc411b51855df0112820fa330baa68908887b8cd0e54b2cab460a794"} Oct 30 18:38:54.874473 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.874430 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67486b6598-97544" event={"ID":"b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c","Type":"ContainerDied","Data":"bb84c7e691d1a2e0737148c677bda973e56154351b6af3f2a31852086ebe28b4"} Oct 30 18:38:54.874473 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.874458 5054 scope.go:122] "RemoveContainer" containerID="7a77cdd9bc411b51855df0112820fa330baa68908887b8cd0e54b2cab460a794" Oct 30 18:38:54.882418 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.882397 5054 scope.go:122] "RemoveContainer" containerID="7a77cdd9bc411b51855df0112820fa330baa68908887b8cd0e54b2cab460a794" Oct 30 18:38:54.882667 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:38:54.882648 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a77cdd9bc411b51855df0112820fa330baa68908887b8cd0e54b2cab460a794\": container with ID starting with 7a77cdd9bc411b51855df0112820fa330baa68908887b8cd0e54b2cab460a794 not found: ID does not exist" containerID="7a77cdd9bc411b51855df0112820fa330baa68908887b8cd0e54b2cab460a794" Oct 30 18:38:54.882712 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:54.882696 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a77cdd9bc411b51855df0112820fa330baa68908887b8cd0e54b2cab460a794"} err="failed to get container status \"7a77cdd9bc411b51855df0112820fa330baa68908887b8cd0e54b2cab460a794\": rpc error: code = NotFound desc = could not find container \"7a77cdd9bc411b51855df0112820fa330baa68908887b8cd0e54b2cab460a794\": container with ID starting with 7a77cdd9bc411b51855df0112820fa330baa68908887b8cd0e54b2cab460a794 not found: ID does not exist" Oct 30 18:38:55.036427 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:55.036397 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67486b6598-97544"] Oct 30 18:38:55.083628 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:55.083596 5054 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67486b6598-97544"] Oct 30 18:38:56.421797 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:56.421763 5054 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c" path="/var/lib/kubelet/pods/b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c/volumes" Oct 30 18:38:58.893509 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:58.893473 5054 generic.go:356] "Generic (PLEG): container finished" podID="00d3be28-5482-48ea-a1be-629e8941236e" containerID="52e79f8bba3ae526daf24de718556caf07d88454dbd395f74b1ccfe79a7b4d81" exitCode=0 Oct 30 18:38:58.893873 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:58.893547 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-q9rgp" event={"ID":"00d3be28-5482-48ea-a1be-629e8941236e","Type":"ContainerDied","Data":"52e79f8bba3ae526daf24de718556caf07d88454dbd395f74b1ccfe79a7b4d81"} Oct 30 18:38:58.893873 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:58.893860 5054 scope.go:122] "RemoveContainer" containerID="52e79f8bba3ae526daf24de718556caf07d88454dbd395f74b1ccfe79a7b4d81" Oct 30 18:38:59.898902 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:59.898869 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-q9rgp" event={"ID":"00d3be28-5482-48ea-a1be-629e8941236e","Type":"ContainerStarted","Data":"84d64672416a150ec40bb7170df167186bed5944a03492bbfca312c9e3a18f0c"} Oct 30 18:38:59.930856 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:59.930825 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-lnkcs-v7pdj"] Oct 30 18:38:59.931030 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:38:59.930993 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" podUID="03157316-915c-412d-9a93-5edcf607a91f" containerName="insights-gathering" containerID="cri-o://cb2f4e8848a187011f38af6c5bc17c6f96a82ab02636a0a8511074a7a330fffd" gracePeriod=30 Oct 30 18:39:00.244139 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:00.244110 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:39:00.342140 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:00.342107 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/03157316-915c-412d-9a93-5edcf607a91f-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03157316-915c-412d-9a93-5edcf607a91f-service-ca-bundle\") pod \"03157316-915c-412d-9a93-5edcf607a91f\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " Oct 30 18:39:00.342298 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:00.342149 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03157316-915c-412d-9a93-5edcf607a91f-kube-api-access-g4bc7\" (UniqueName: \"kubernetes.io/projected/03157316-915c-412d-9a93-5edcf607a91f-kube-api-access-g4bc7\") pod \"03157316-915c-412d-9a93-5edcf607a91f\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " Oct 30 18:39:00.342298 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:00.342200 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/03157316-915c-412d-9a93-5edcf607a91f-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/03157316-915c-412d-9a93-5edcf607a91f-archives-path\") pod \"03157316-915c-412d-9a93-5edcf607a91f\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " Oct 30 18:39:00.342298 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:00.342231 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/03157316-915c-412d-9a93-5edcf607a91f-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03157316-915c-412d-9a93-5edcf607a91f-trusted-ca-bundle\") pod \"03157316-915c-412d-9a93-5edcf607a91f\" (UID: \"03157316-915c-412d-9a93-5edcf607a91f\") " Oct 30 18:39:00.342545 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:00.342519 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03157316-915c-412d-9a93-5edcf607a91f-archives-path" pod "03157316-915c-412d-9a93-5edcf607a91f" (UID: "03157316-915c-412d-9a93-5edcf607a91f"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:39:00.342616 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:00.342550 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03157316-915c-412d-9a93-5edcf607a91f-service-ca-bundle" pod "03157316-915c-412d-9a93-5edcf607a91f" (UID: "03157316-915c-412d-9a93-5edcf607a91f"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:39:00.342654 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:00.342628 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03157316-915c-412d-9a93-5edcf607a91f-trusted-ca-bundle" pod "03157316-915c-412d-9a93-5edcf607a91f" (UID: "03157316-915c-412d-9a93-5edcf607a91f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:39:00.344593 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:00.344572 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03157316-915c-412d-9a93-5edcf607a91f-kube-api-access-g4bc7" pod "03157316-915c-412d-9a93-5edcf607a91f" (UID: "03157316-915c-412d-9a93-5edcf607a91f"). InnerVolumeSpecName "kube-api-access-g4bc7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:39:00.443109 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:00.443080 5054 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/03157316-915c-412d-9a93-5edcf607a91f-archives-path\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:39:00.443109 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:00.443104 5054 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03157316-915c-412d-9a93-5edcf607a91f-trusted-ca-bundle\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:39:00.443109 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:00.443114 5054 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03157316-915c-412d-9a93-5edcf607a91f-service-ca-bundle\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:39:00.443301 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:00.443125 5054 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4bc7\" (UniqueName: \"kubernetes.io/projected/03157316-915c-412d-9a93-5edcf607a91f-kube-api-access-g4bc7\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:39:00.903769 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:00.903733 5054 generic.go:356] "Generic (PLEG): container finished" podID="03157316-915c-412d-9a93-5edcf607a91f" containerID="cb2f4e8848a187011f38af6c5bc17c6f96a82ab02636a0a8511074a7a330fffd" exitCode=2 Oct 30 18:39:00.904210 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:00.903786 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" Oct 30 18:39:00.904210 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:00.903824 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" event={"ID":"03157316-915c-412d-9a93-5edcf607a91f","Type":"ContainerDied","Data":"cb2f4e8848a187011f38af6c5bc17c6f96a82ab02636a0a8511074a7a330fffd"} Oct 30 18:39:00.904210 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:00.903873 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-lnkcs-v7pdj" event={"ID":"03157316-915c-412d-9a93-5edcf607a91f","Type":"ContainerDied","Data":"53a0a315d1ae630f72c402a412e936660f22e80450846e2c9d0a0a05ecf09d35"} Oct 30 18:39:00.904210 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:00.903894 5054 scope.go:122] "RemoveContainer" containerID="cb2f4e8848a187011f38af6c5bc17c6f96a82ab02636a0a8511074a7a330fffd" Oct 30 18:39:00.911874 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:00.911848 5054 scope.go:122] "RemoveContainer" containerID="cb2f4e8848a187011f38af6c5bc17c6f96a82ab02636a0a8511074a7a330fffd" Oct 30 18:39:00.912138 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:39:00.912117 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb2f4e8848a187011f38af6c5bc17c6f96a82ab02636a0a8511074a7a330fffd\": container with ID starting with cb2f4e8848a187011f38af6c5bc17c6f96a82ab02636a0a8511074a7a330fffd not found: ID does not exist" containerID="cb2f4e8848a187011f38af6c5bc17c6f96a82ab02636a0a8511074a7a330fffd" Oct 30 18:39:00.912214 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:00.912146 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb2f4e8848a187011f38af6c5bc17c6f96a82ab02636a0a8511074a7a330fffd"} err="failed to get container status \"cb2f4e8848a187011f38af6c5bc17c6f96a82ab02636a0a8511074a7a330fffd\": rpc error: code = NotFound desc = could not find container \"cb2f4e8848a187011f38af6c5bc17c6f96a82ab02636a0a8511074a7a330fffd\": container with ID starting with cb2f4e8848a187011f38af6c5bc17c6f96a82ab02636a0a8511074a7a330fffd not found: ID does not exist" Oct 30 18:39:01.069465 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:01.069435 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-lnkcs-v7pdj"] Oct 30 18:39:01.075403 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:01.075380 5054 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-lnkcs-v7pdj"] Oct 30 18:39:02.421187 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:02.421155 5054 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="03157316-915c-412d-9a93-5edcf607a91f" path="/var/lib/kubelet/pods/03157316-915c-412d-9a93-5edcf607a91f/volumes" Oct 30 18:39:08.596391 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:39:08.596353 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:39:11.861456 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.861423 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-56b846fc8c-d27m6"] Oct 30 18:39:11.861824 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.861730 5054 cpu_manager.go:414] "containerMap: removing container" podUID="03157316-915c-412d-9a93-5edcf607a91f" containerName="insights-gathering" Oct 30 18:39:11.861824 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.861741 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="03157316-915c-412d-9a93-5edcf607a91f" containerName="insights-gathering" Oct 30 18:39:11.861824 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.861754 5054 cpu_manager.go:414] "containerMap: removing container" podUID="b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c" containerName="console" Oct 30 18:39:11.861824 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.861759 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c" containerName="console" Oct 30 18:39:11.861824 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.861778 5054 cpu_manager.go:414] "containerMap: removing container" podUID="d9b465b9-97b6-47e2-8732-ed29b8f24ed0" containerName="registry" Oct 30 18:39:11.861824 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.861783 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9b465b9-97b6-47e2-8732-ed29b8f24ed0" containerName="registry" Oct 30 18:39:11.861824 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.861797 5054 cpu_manager.go:414] "containerMap: removing container" podUID="7fcdea8f-3b0f-42c5-bb3d-b2266b046491" containerName="registry" Oct 30 18:39:11.861824 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.861802 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7fcdea8f-3b0f-42c5-bb3d-b2266b046491" containerName="registry" Oct 30 18:39:11.862096 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.861852 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="03157316-915c-412d-9a93-5edcf607a91f" containerName="insights-gathering" Oct 30 18:39:11.862096 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.861861 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="b83b2ad9-2bf1-4e61-b0b8-5c72761bd34c" containerName="console" Oct 30 18:39:11.862096 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.861869 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9b465b9-97b6-47e2-8732-ed29b8f24ed0" containerName="registry" Oct 30 18:39:11.862096 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.861877 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="7fcdea8f-3b0f-42c5-bb3d-b2266b046491" containerName="registry" Oct 30 18:39:11.865808 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.865786 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:11.936363 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.936335 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-oauth-serving-cert\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:11.936509 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.936369 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d779d9c2-2c92-4a8b-9749-74c3721563a5-console-serving-cert\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:11.936509 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.936411 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-service-ca\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:11.936645 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.936544 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d779d9c2-2c92-4a8b-9749-74c3721563a5-console-oauth-config\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:11.936645 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.936586 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-trusted-ca-bundle\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:11.936645 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.936618 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d779d9c2-2c92-4a8b-9749-74c3721563a5-tmp\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:11.936758 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.936701 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-console-config\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:11.936758 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.936725 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4k7b\" (UniqueName: \"kubernetes.io/projected/d779d9c2-2c92-4a8b-9749-74c3721563a5-kube-api-access-k4k7b\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:11.965771 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:11.965743 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56b846fc8c-d27m6"] Oct 30 18:39:12.037668 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:12.037642 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-oauth-serving-cert\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:12.037830 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:12.037675 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d779d9c2-2c92-4a8b-9749-74c3721563a5-console-serving-cert\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:12.037830 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:12.037704 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-service-ca\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:12.037830 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:12.037747 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d779d9c2-2c92-4a8b-9749-74c3721563a5-console-oauth-config\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:12.037830 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:12.037775 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-trusted-ca-bundle\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:12.037830 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:12.037800 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d779d9c2-2c92-4a8b-9749-74c3721563a5-tmp\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:12.037830 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:12.037826 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-console-config\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:12.038176 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:12.037857 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4k7b\" (UniqueName: \"kubernetes.io/projected/d779d9c2-2c92-4a8b-9749-74c3721563a5-kube-api-access-k4k7b\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:12.038505 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:12.038483 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-service-ca\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:12.038616 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:12.038525 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d779d9c2-2c92-4a8b-9749-74c3721563a5-tmp\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:12.038677 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:12.038548 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-oauth-serving-cert\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:12.038677 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:12.038572 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-console-config\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:12.039097 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:12.039073 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-trusted-ca-bundle\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:12.040230 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:12.040208 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d779d9c2-2c92-4a8b-9749-74c3721563a5-console-oauth-config\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:12.040428 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:12.040411 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d779d9c2-2c92-4a8b-9749-74c3721563a5-console-serving-cert\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:12.118087 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:12.118023 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4k7b\" (UniqueName: \"kubernetes.io/projected/d779d9c2-2c92-4a8b-9749-74c3721563a5-kube-api-access-k4k7b\") pod \"console-56b846fc8c-d27m6\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:12.177190 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:12.177162 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:12.553151 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:12.553079 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56b846fc8c-d27m6"] Oct 30 18:39:12.556891 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:39:12.556867 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd779d9c2_2c92_4a8b_9749_74c3721563a5.slice/crio-127b06db71275feb47591d4702e26f0f9663dd1e20a7a9c58b71fabe4665eaab WatchSource:0}: Error finding container 127b06db71275feb47591d4702e26f0f9663dd1e20a7a9c58b71fabe4665eaab: Status 404 returned error can't find the container with id 127b06db71275feb47591d4702e26f0f9663dd1e20a7a9c58b71fabe4665eaab Oct 30 18:39:12.957363 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:12.957328 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56b846fc8c-d27m6" event={"ID":"d779d9c2-2c92-4a8b-9749-74c3721563a5","Type":"ContainerStarted","Data":"b0a3df81d39d2b9fc22bc1dcf0587504fd861a837a1ffb8d10deb60015a8de4b"} Oct 30 18:39:12.957857 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:12.957375 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56b846fc8c-d27m6" event={"ID":"d779d9c2-2c92-4a8b-9749-74c3721563a5","Type":"ContainerStarted","Data":"127b06db71275feb47591d4702e26f0f9663dd1e20a7a9c58b71fabe4665eaab"} Oct 30 18:39:13.076320 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:13.076271 5054 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-56b846fc8c-d27m6" podStartSLOduration=2.076255963 podStartE2EDuration="2.076255963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:39:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:39:13.074360893 +0000 UTC m=+94.802739610" watchObservedRunningTime="2026-10-30 18:39:13.076255963 +0000 UTC m=+94.804634682" Oct 30 18:39:22.177690 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:22.177658 5054 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:22.178177 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:22.177711 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:22.182701 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:22.182680 5054 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:23.000025 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:22.999984 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:39:23.169863 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:23.169826 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-568f944-hrvh5"] Oct 30 18:39:38.599537 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:39:38.599501 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:39:48.190895 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.190837 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-568f944-hrvh5" podUID="a63d77ef-dc01-4e7d-bb48-7657f2e7d626" containerName="console" containerID="cri-o://f8ebf6873deefd161e1e09ae2167898eaa96dbcc11b6dfc0eb8f509d411f0f56" gracePeriod=15 Oct 30 18:39:48.463802 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.463776 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-568f944-hrvh5" Oct 30 18:39:48.639434 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.639355 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-console-serving-cert\") pod \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " Oct 30 18:39:48.639434 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.639406 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-oauth-serving-cert\") pod \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " Oct 30 18:39:48.639434 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.639425 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-trusted-ca-bundle\") pod \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " Oct 30 18:39:48.639713 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.639476 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-console-oauth-config\") pod \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " Oct 30 18:39:48.639713 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.639503 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-console-config\" (UniqueName: \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-console-config\") pod \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " Oct 30 18:39:48.639713 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.639532 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-kube-api-access-24lrn\" (UniqueName: \"kubernetes.io/projected/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-kube-api-access-24lrn\") pod \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " Oct 30 18:39:48.639713 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.639594 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-tmp\" (UniqueName: \"kubernetes.io/empty-dir/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-tmp\") pod \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " Oct 30 18:39:48.639713 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.639621 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-service-ca\" (UniqueName: \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-service-ca\") pod \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\" (UID: \"a63d77ef-dc01-4e7d-bb48-7657f2e7d626\") " Oct 30 18:39:48.640057 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.639921 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-oauth-serving-cert" pod "a63d77ef-dc01-4e7d-bb48-7657f2e7d626" (UID: "a63d77ef-dc01-4e7d-bb48-7657f2e7d626"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:39:48.640057 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.639940 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-console-config" pod "a63d77ef-dc01-4e7d-bb48-7657f2e7d626" (UID: "a63d77ef-dc01-4e7d-bb48-7657f2e7d626"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:39:48.640057 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.639927 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-trusted-ca-bundle" pod "a63d77ef-dc01-4e7d-bb48-7657f2e7d626" (UID: "a63d77ef-dc01-4e7d-bb48-7657f2e7d626"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:39:48.640205 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.640151 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-tmp" pod "a63d77ef-dc01-4e7d-bb48-7657f2e7d626" (UID: "a63d77ef-dc01-4e7d-bb48-7657f2e7d626"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:39:48.640205 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.640161 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-service-ca" pod "a63d77ef-dc01-4e7d-bb48-7657f2e7d626" (UID: "a63d77ef-dc01-4e7d-bb48-7657f2e7d626"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:39:48.641644 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.641617 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-console-oauth-config" pod "a63d77ef-dc01-4e7d-bb48-7657f2e7d626" (UID: "a63d77ef-dc01-4e7d-bb48-7657f2e7d626"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:39:48.641750 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.641660 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-console-serving-cert" pod "a63d77ef-dc01-4e7d-bb48-7657f2e7d626" (UID: "a63d77ef-dc01-4e7d-bb48-7657f2e7d626"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:39:48.641750 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.641701 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-kube-api-access-24lrn" pod "a63d77ef-dc01-4e7d-bb48-7657f2e7d626" (UID: "a63d77ef-dc01-4e7d-bb48-7657f2e7d626"). InnerVolumeSpecName "kube-api-access-24lrn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:39:48.741173 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.741101 5054 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-oauth-serving-cert\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:39:48.741173 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.741126 5054 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-trusted-ca-bundle\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:39:48.741173 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.741136 5054 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-console-oauth-config\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:39:48.741173 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.741146 5054 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-console-config\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:39:48.741173 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.741154 5054 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24lrn\" (UniqueName: \"kubernetes.io/projected/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-kube-api-access-24lrn\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:39:48.741173 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.741165 5054 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-tmp\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:39:48.741173 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.741174 5054 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-service-ca\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:39:48.741602 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:48.741182 5054 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a63d77ef-dc01-4e7d-bb48-7657f2e7d626-console-serving-cert\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:39:49.099891 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:49.099815 5054 generic.go:356] "Generic (PLEG): container finished" podID="a63d77ef-dc01-4e7d-bb48-7657f2e7d626" containerID="f8ebf6873deefd161e1e09ae2167898eaa96dbcc11b6dfc0eb8f509d411f0f56" exitCode=0 Oct 30 18:39:49.099891 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:49.099858 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-568f944-hrvh5" event={"ID":"a63d77ef-dc01-4e7d-bb48-7657f2e7d626","Type":"ContainerDied","Data":"f8ebf6873deefd161e1e09ae2167898eaa96dbcc11b6dfc0eb8f509d411f0f56"} Oct 30 18:39:49.100099 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:49.099893 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-568f944-hrvh5" event={"ID":"a63d77ef-dc01-4e7d-bb48-7657f2e7d626","Type":"ContainerDied","Data":"b82fd16a8e9ab6b4302e8b8ebaebbbde35bf07ac545ae7a16a5b1fb56294533a"} Oct 30 18:39:49.100099 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:49.099912 5054 scope.go:122] "RemoveContainer" containerID="f8ebf6873deefd161e1e09ae2167898eaa96dbcc11b6dfc0eb8f509d411f0f56" Oct 30 18:39:49.100099 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:49.099860 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-568f944-hrvh5" Oct 30 18:39:49.108681 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:49.108666 5054 scope.go:122] "RemoveContainer" containerID="f8ebf6873deefd161e1e09ae2167898eaa96dbcc11b6dfc0eb8f509d411f0f56" Oct 30 18:39:49.108955 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:39:49.108932 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8ebf6873deefd161e1e09ae2167898eaa96dbcc11b6dfc0eb8f509d411f0f56\": container with ID starting with f8ebf6873deefd161e1e09ae2167898eaa96dbcc11b6dfc0eb8f509d411f0f56 not found: ID does not exist" containerID="f8ebf6873deefd161e1e09ae2167898eaa96dbcc11b6dfc0eb8f509d411f0f56" Oct 30 18:39:49.109072 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:49.108961 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8ebf6873deefd161e1e09ae2167898eaa96dbcc11b6dfc0eb8f509d411f0f56"} err="failed to get container status \"f8ebf6873deefd161e1e09ae2167898eaa96dbcc11b6dfc0eb8f509d411f0f56\": rpc error: code = NotFound desc = could not find container \"f8ebf6873deefd161e1e09ae2167898eaa96dbcc11b6dfc0eb8f509d411f0f56\": container with ID starting with f8ebf6873deefd161e1e09ae2167898eaa96dbcc11b6dfc0eb8f509d411f0f56 not found: ID does not exist" Oct 30 18:39:49.289543 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:49.289517 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-568f944-hrvh5"] Oct 30 18:39:49.336601 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:49.336575 5054 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-568f944-hrvh5"] Oct 30 18:39:50.421915 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:39:50.421885 5054 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a63d77ef-dc01-4e7d-bb48-7657f2e7d626" path="/var/lib/kubelet/pods/a63d77ef-dc01-4e7d-bb48-7657f2e7d626/volumes" Oct 30 18:40:08.596845 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:40:08.596803 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:40:38.596161 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:40:38.596125 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:40:41.702912 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.702874 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d54c54855-kffjb"] Oct 30 18:40:41.703389 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.703222 5054 cpu_manager.go:414] "containerMap: removing container" podUID="a63d77ef-dc01-4e7d-bb48-7657f2e7d626" containerName="console" Oct 30 18:40:41.703389 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.703233 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a63d77ef-dc01-4e7d-bb48-7657f2e7d626" containerName="console" Oct 30 18:40:41.703389 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.703291 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="a63d77ef-dc01-4e7d-bb48-7657f2e7d626" containerName="console" Oct 30 18:40:41.707391 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.707370 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.807095 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.807063 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d54c54855-kffjb"] Oct 30 18:40:41.846557 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.846520 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9b33fd2f-5244-4df8-9037-b577a80b560c-console-oauth-config\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.846557 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.846555 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9b33fd2f-5244-4df8-9037-b577a80b560c-console-config\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.846710 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.846575 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9b33fd2f-5244-4df8-9037-b577a80b560c-service-ca\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.846710 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.846638 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9b33fd2f-5244-4df8-9037-b577a80b560c-oauth-serving-cert\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.846710 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.846669 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv5qn\" (UniqueName: \"kubernetes.io/projected/9b33fd2f-5244-4df8-9037-b577a80b560c-kube-api-access-pv5qn\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.846807 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.846745 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b33fd2f-5244-4df8-9037-b577a80b560c-trusted-ca-bundle\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.846807 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.846801 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9b33fd2f-5244-4df8-9037-b577a80b560c-console-serving-cert\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.846868 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.846819 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b33fd2f-5244-4df8-9037-b577a80b560c-tmp\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.947239 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.947208 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9b33fd2f-5244-4df8-9037-b577a80b560c-console-oauth-config\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.947239 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.947239 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9b33fd2f-5244-4df8-9037-b577a80b560c-console-config\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.947429 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.947256 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9b33fd2f-5244-4df8-9037-b577a80b560c-service-ca\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.947429 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.947273 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9b33fd2f-5244-4df8-9037-b577a80b560c-oauth-serving-cert\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.947429 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.947290 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv5qn\" (UniqueName: \"kubernetes.io/projected/9b33fd2f-5244-4df8-9037-b577a80b560c-kube-api-access-pv5qn\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.947429 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.947338 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b33fd2f-5244-4df8-9037-b577a80b560c-trusted-ca-bundle\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.947429 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.947375 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9b33fd2f-5244-4df8-9037-b577a80b560c-console-serving-cert\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.947429 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.947397 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b33fd2f-5244-4df8-9037-b577a80b560c-tmp\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.947994 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.947965 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9b33fd2f-5244-4df8-9037-b577a80b560c-tmp\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.948132 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.948112 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9b33fd2f-5244-4df8-9037-b577a80b560c-service-ca\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.948199 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.948163 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9b33fd2f-5244-4df8-9037-b577a80b560c-console-config\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.948254 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.948238 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9b33fd2f-5244-4df8-9037-b577a80b560c-oauth-serving-cert\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.948567 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.948540 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b33fd2f-5244-4df8-9037-b577a80b560c-trusted-ca-bundle\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.949952 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.949928 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9b33fd2f-5244-4df8-9037-b577a80b560c-console-oauth-config\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:41.949952 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:41.949942 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9b33fd2f-5244-4df8-9037-b577a80b560c-console-serving-cert\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:42.008157 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:42.008092 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv5qn\" (UniqueName: \"kubernetes.io/projected/9b33fd2f-5244-4df8-9037-b577a80b560c-kube-api-access-pv5qn\") pod \"console-5d54c54855-kffjb\" (UID: \"9b33fd2f-5244-4df8-9037-b577a80b560c\") " pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:42.018521 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:42.018490 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:42.438088 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:42.438055 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d54c54855-kffjb"] Oct 30 18:40:42.444860 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:40:42.444825 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b33fd2f_5244_4df8_9037_b577a80b560c.slice/crio-b167d46a6eec54e4c6a6a123efb362eb73ef247a15d1bd47a88af13a1bf1cfbb WatchSource:0}: Error finding container b167d46a6eec54e4c6a6a123efb362eb73ef247a15d1bd47a88af13a1bf1cfbb: Status 404 returned error can't find the container with id b167d46a6eec54e4c6a6a123efb362eb73ef247a15d1bd47a88af13a1bf1cfbb Oct 30 18:40:43.315648 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:43.315613 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d54c54855-kffjb" event={"ID":"9b33fd2f-5244-4df8-9037-b577a80b560c","Type":"ContainerStarted","Data":"b8a8a1681045140bfaea80b04c20190d668af21589f5b243f78b6eaf21d0e3de"} Oct 30 18:40:43.316071 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:43.315663 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d54c54855-kffjb" event={"ID":"9b33fd2f-5244-4df8-9037-b577a80b560c","Type":"ContainerStarted","Data":"b167d46a6eec54e4c6a6a123efb362eb73ef247a15d1bd47a88af13a1bf1cfbb"} Oct 30 18:40:43.476669 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:43.476622 5054 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-5d54c54855-kffjb" podStartSLOduration=2.4766093 podStartE2EDuration="2.4766093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:40:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:40:43.475526996 +0000 UTC m=+185.203905726" watchObservedRunningTime="2026-10-30 18:40:43.4766093 +0000 UTC m=+185.204988007" Oct 30 18:40:52.019593 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:52.019544 5054 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:52.019969 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:52.019609 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:52.024278 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:52.024254 5054 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:52.355432 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:52.355357 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d54c54855-kffjb" Oct 30 18:40:52.507289 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:40:52.507256 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56b846fc8c-d27m6"] Oct 30 18:41:08.596915 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:41:08.596879 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:41:18.374300 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.374261 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-56b846fc8c-d27m6" podUID="d779d9c2-2c92-4a8b-9749-74c3721563a5" containerName="console" containerID="cri-o://b0a3df81d39d2b9fc22bc1dcf0587504fd861a837a1ffb8d10deb60015a8de4b" gracePeriod=15 Oct 30 18:41:18.618977 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.618953 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:41:18.746849 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.746747 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-oauth-serving-cert\") pod \"d779d9c2-2c92-4a8b-9749-74c3721563a5\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " Oct 30 18:41:18.746849 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.746786 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d779d9c2-2c92-4a8b-9749-74c3721563a5-kube-api-access-k4k7b\" (UniqueName: \"kubernetes.io/projected/d779d9c2-2c92-4a8b-9749-74c3721563a5-kube-api-access-k4k7b\") pod \"d779d9c2-2c92-4a8b-9749-74c3721563a5\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " Oct 30 18:41:18.746849 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.746823 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d779d9c2-2c92-4a8b-9749-74c3721563a5-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d779d9c2-2c92-4a8b-9749-74c3721563a5-console-oauth-config\") pod \"d779d9c2-2c92-4a8b-9749-74c3721563a5\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " Oct 30 18:41:18.747136 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.747036 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d779d9c2-2c92-4a8b-9749-74c3721563a5-tmp\" (UniqueName: \"kubernetes.io/empty-dir/d779d9c2-2c92-4a8b-9749-74c3721563a5-tmp\") pod \"d779d9c2-2c92-4a8b-9749-74c3721563a5\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " Oct 30 18:41:18.747136 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.747124 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-service-ca\" (UniqueName: \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-service-ca\") pod \"d779d9c2-2c92-4a8b-9749-74c3721563a5\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " Oct 30 18:41:18.747247 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.747166 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-console-config\" (UniqueName: \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-console-config\") pod \"d779d9c2-2c92-4a8b-9749-74c3721563a5\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " Oct 30 18:41:18.747247 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.747186 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-oauth-serving-cert" pod "d779d9c2-2c92-4a8b-9749-74c3721563a5" (UID: "d779d9c2-2c92-4a8b-9749-74c3721563a5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:18.747247 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.747207 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d779d9c2-2c92-4a8b-9749-74c3721563a5-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d779d9c2-2c92-4a8b-9749-74c3721563a5-console-serving-cert\") pod \"d779d9c2-2c92-4a8b-9749-74c3721563a5\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " Oct 30 18:41:18.747402 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.747246 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-trusted-ca-bundle\") pod \"d779d9c2-2c92-4a8b-9749-74c3721563a5\" (UID: \"d779d9c2-2c92-4a8b-9749-74c3721563a5\") " Oct 30 18:41:18.747402 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.747304 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d779d9c2-2c92-4a8b-9749-74c3721563a5-tmp" pod "d779d9c2-2c92-4a8b-9749-74c3721563a5" (UID: "d779d9c2-2c92-4a8b-9749-74c3721563a5"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:41:18.747592 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.747535 5054 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d779d9c2-2c92-4a8b-9749-74c3721563a5-tmp\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:41:18.747592 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.747561 5054 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-oauth-serving-cert\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:41:18.747773 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.747591 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-console-config" pod "d779d9c2-2c92-4a8b-9749-74c3721563a5" (UID: "d779d9c2-2c92-4a8b-9749-74c3721563a5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:18.747773 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.747684 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-trusted-ca-bundle" pod "d779d9c2-2c92-4a8b-9749-74c3721563a5" (UID: "d779d9c2-2c92-4a8b-9749-74c3721563a5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:18.747898 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.747793 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-service-ca" pod "d779d9c2-2c92-4a8b-9749-74c3721563a5" (UID: "d779d9c2-2c92-4a8b-9749-74c3721563a5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:41:18.749067 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.749044 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d779d9c2-2c92-4a8b-9749-74c3721563a5-kube-api-access-k4k7b" pod "d779d9c2-2c92-4a8b-9749-74c3721563a5" (UID: "d779d9c2-2c92-4a8b-9749-74c3721563a5"). InnerVolumeSpecName "kube-api-access-k4k7b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:41:18.749196 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.749180 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d779d9c2-2c92-4a8b-9749-74c3721563a5-console-oauth-config" pod "d779d9c2-2c92-4a8b-9749-74c3721563a5" (UID: "d779d9c2-2c92-4a8b-9749-74c3721563a5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:41:18.749358 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.749334 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d779d9c2-2c92-4a8b-9749-74c3721563a5-console-serving-cert" pod "d779d9c2-2c92-4a8b-9749-74c3721563a5" (UID: "d779d9c2-2c92-4a8b-9749-74c3721563a5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:41:18.848559 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.848532 5054 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4k7b\" (UniqueName: \"kubernetes.io/projected/d779d9c2-2c92-4a8b-9749-74c3721563a5-kube-api-access-k4k7b\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:41:18.848559 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.848560 5054 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d779d9c2-2c92-4a8b-9749-74c3721563a5-console-oauth-config\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:41:18.848718 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.848575 5054 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-service-ca\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:41:18.848718 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.848586 5054 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-console-config\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:41:18.848718 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.848599 5054 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d779d9c2-2c92-4a8b-9749-74c3721563a5-console-serving-cert\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:41:18.848718 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:18.848611 5054 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d779d9c2-2c92-4a8b-9749-74c3721563a5-trusted-ca-bundle\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:41:19.458190 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:19.458152 5054 generic.go:356] "Generic (PLEG): container finished" podID="d779d9c2-2c92-4a8b-9749-74c3721563a5" containerID="b0a3df81d39d2b9fc22bc1dcf0587504fd861a837a1ffb8d10deb60015a8de4b" exitCode=0 Oct 30 18:41:19.458577 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:19.458199 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56b846fc8c-d27m6" event={"ID":"d779d9c2-2c92-4a8b-9749-74c3721563a5","Type":"ContainerDied","Data":"b0a3df81d39d2b9fc22bc1dcf0587504fd861a837a1ffb8d10deb60015a8de4b"} Oct 30 18:41:19.458577 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:19.458208 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56b846fc8c-d27m6" Oct 30 18:41:19.458577 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:19.458245 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56b846fc8c-d27m6" event={"ID":"d779d9c2-2c92-4a8b-9749-74c3721563a5","Type":"ContainerDied","Data":"127b06db71275feb47591d4702e26f0f9663dd1e20a7a9c58b71fabe4665eaab"} Oct 30 18:41:19.458577 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:19.458271 5054 scope.go:122] "RemoveContainer" containerID="b0a3df81d39d2b9fc22bc1dcf0587504fd861a837a1ffb8d10deb60015a8de4b" Oct 30 18:41:19.467439 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:19.467422 5054 scope.go:122] "RemoveContainer" containerID="b0a3df81d39d2b9fc22bc1dcf0587504fd861a837a1ffb8d10deb60015a8de4b" Oct 30 18:41:19.467702 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:41:19.467685 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0a3df81d39d2b9fc22bc1dcf0587504fd861a837a1ffb8d10deb60015a8de4b\": container with ID starting with b0a3df81d39d2b9fc22bc1dcf0587504fd861a837a1ffb8d10deb60015a8de4b not found: ID does not exist" containerID="b0a3df81d39d2b9fc22bc1dcf0587504fd861a837a1ffb8d10deb60015a8de4b" Oct 30 18:41:19.467736 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:19.467711 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0a3df81d39d2b9fc22bc1dcf0587504fd861a837a1ffb8d10deb60015a8de4b"} err="failed to get container status \"b0a3df81d39d2b9fc22bc1dcf0587504fd861a837a1ffb8d10deb60015a8de4b\": rpc error: code = NotFound desc = could not find container \"b0a3df81d39d2b9fc22bc1dcf0587504fd861a837a1ffb8d10deb60015a8de4b\": container with ID starting with b0a3df81d39d2b9fc22bc1dcf0587504fd861a837a1ffb8d10deb60015a8de4b not found: ID does not exist" Oct 30 18:41:19.624762 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:19.624727 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56b846fc8c-d27m6"] Oct 30 18:41:19.672701 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:19.672670 5054 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56b846fc8c-d27m6"] Oct 30 18:41:20.421664 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:41:20.421631 5054 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d779d9c2-2c92-4a8b-9749-74c3721563a5" path="/var/lib/kubelet/pods/d779d9c2-2c92-4a8b-9749-74c3721563a5/volumes" Oct 30 18:41:38.599244 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:41:38.599209 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:42:08.598803 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:42:08.598769 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:42:38.442806 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:42:38.442774 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 18:42:38.443944 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:42:38.443916 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 18:42:38.598634 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:42:38.598594 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:43:08.598075 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:43:08.598039 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:43:38.597067 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:43:38.597033 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:44:08.597151 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:44:08.597111 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:44:38.597483 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:44:38.597446 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:45:08.597350 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:45:08.597317 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:45:38.596689 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:45:38.596653 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:46:08.597209 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:46:08.597175 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:46:38.596857 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:46:38.596817 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:47:08.596480 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:47:08.596441 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:47:38.467325 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:47:38.467296 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 18:47:38.468526 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:47:38.468499 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 18:47:38.596758 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:47:38.596722 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:48:08.596499 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:48:08.596460 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:48:38.596461 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:48:38.596421 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:49:08.596713 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:49:08.596676 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:49:38.595762 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:49:38.595722 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:50:00.035517 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.035483 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz"] Oct 30 18:50:00.035908 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.035822 5054 cpu_manager.go:414] "containerMap: removing container" podUID="d779d9c2-2c92-4a8b-9749-74c3721563a5" containerName="console" Oct 30 18:50:00.035908 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.035833 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d779d9c2-2c92-4a8b-9749-74c3721563a5" containerName="console" Oct 30 18:50:00.035987 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.035914 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="d779d9c2-2c92-4a8b-9749-74c3721563a5" containerName="console" Oct 30 18:50:00.039076 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.039059 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:50:00.091861 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.091838 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"isvc-canary-rollback-kube-rbac-proxy-sar-config\"" Oct 30 18:50:00.092163 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.092144 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"seaweedfs-s3-creds\"" Oct 30 18:50:00.138684 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.138653 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"kube-root-ca.crt\"" Oct 30 18:50:00.138905 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.138885 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"openshift-service-ca.crt\"" Oct 30 18:50:00.138966 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.138901 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"default-dockercfg-zc4kv\"" Oct 30 18:50:00.138966 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.138900 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"isvc-canary-rollback-predictor-serving-cert\"" Oct 30 18:50:00.138966 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.138889 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"odh-kserve-custom-ca-bundle\"" Oct 30 18:50:00.140491 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.140474 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz"] Oct 30 18:50:00.156548 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.156518 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f1b60250-3c6a-4004-8e1e-c54233c44cc1-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-554c7cc56d-j7cqz\" (UID: \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\") " pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:50:00.156646 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.156561 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f1b60250-3c6a-4004-8e1e-c54233c44cc1-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-554c7cc56d-j7cqz\" (UID: \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\") " pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:50:00.156646 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.156593 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jzll\" (UniqueName: \"kubernetes.io/projected/f1b60250-3c6a-4004-8e1e-c54233c44cc1-kube-api-access-7jzll\") pod \"isvc-canary-rollback-predictor-554c7cc56d-j7cqz\" (UID: \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\") " pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:50:00.156725 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.156670 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f1b60250-3c6a-4004-8e1e-c54233c44cc1-proxy-tls\") pod \"isvc-canary-rollback-predictor-554c7cc56d-j7cqz\" (UID: \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\") " pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:50:00.156725 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.156708 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f1b60250-3c6a-4004-8e1e-c54233c44cc1-cabundle-cert\") pod \"isvc-canary-rollback-predictor-554c7cc56d-j7cqz\" (UID: \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\") " pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:50:00.257596 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.257567 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f1b60250-3c6a-4004-8e1e-c54233c44cc1-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-554c7cc56d-j7cqz\" (UID: \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\") " pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:50:00.257710 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.257605 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f1b60250-3c6a-4004-8e1e-c54233c44cc1-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-554c7cc56d-j7cqz\" (UID: \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\") " pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:50:00.257710 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.257632 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jzll\" (UniqueName: \"kubernetes.io/projected/f1b60250-3c6a-4004-8e1e-c54233c44cc1-kube-api-access-7jzll\") pod \"isvc-canary-rollback-predictor-554c7cc56d-j7cqz\" (UID: \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\") " pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:50:00.257710 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.257661 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f1b60250-3c6a-4004-8e1e-c54233c44cc1-proxy-tls\") pod \"isvc-canary-rollback-predictor-554c7cc56d-j7cqz\" (UID: \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\") " pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:50:00.257710 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.257701 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f1b60250-3c6a-4004-8e1e-c54233c44cc1-cabundle-cert\") pod \"isvc-canary-rollback-predictor-554c7cc56d-j7cqz\" (UID: \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\") " pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:50:00.258091 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.258067 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f1b60250-3c6a-4004-8e1e-c54233c44cc1-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-554c7cc56d-j7cqz\" (UID: \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\") " pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:50:00.258440 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.258420 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f1b60250-3c6a-4004-8e1e-c54233c44cc1-cabundle-cert\") pod \"isvc-canary-rollback-predictor-554c7cc56d-j7cqz\" (UID: \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\") " pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:50:00.258487 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.258420 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f1b60250-3c6a-4004-8e1e-c54233c44cc1-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-554c7cc56d-j7cqz\" (UID: \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\") " pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:50:00.260058 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.260041 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f1b60250-3c6a-4004-8e1e-c54233c44cc1-proxy-tls\") pod \"isvc-canary-rollback-predictor-554c7cc56d-j7cqz\" (UID: \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\") " pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:50:00.318652 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.318592 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jzll\" (UniqueName: \"kubernetes.io/projected/f1b60250-3c6a-4004-8e1e-c54233c44cc1-kube-api-access-7jzll\") pod \"isvc-canary-rollback-predictor-554c7cc56d-j7cqz\" (UID: \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\") " pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:50:00.356150 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.356125 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:50:00.777332 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.777298 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz"] Oct 30 18:50:00.782550 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:50:00.782516 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1b60250_3c6a_4004_8e1e_c54233c44cc1.slice/crio-e3661f5ac11eb41b10f2901ef9f4fd227ceb8557597b41ee1c849da351dc7d8f WatchSource:0}: Error finding container e3661f5ac11eb41b10f2901ef9f4fd227ceb8557597b41ee1c849da351dc7d8f: Status 404 returned error can't find the container with id e3661f5ac11eb41b10f2901ef9f4fd227ceb8557597b41ee1c849da351dc7d8f Oct 30 18:50:00.784509 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:00.784493 5054 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:50:01.500649 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:01.500613 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" event={"ID":"f1b60250-3c6a-4004-8e1e-c54233c44cc1","Type":"ContainerStarted","Data":"e3661f5ac11eb41b10f2901ef9f4fd227ceb8557597b41ee1c849da351dc7d8f"} Oct 30 18:50:08.600898 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:50:08.600861 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:50:09.537990 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:09.537947 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" event={"ID":"f1b60250-3c6a-4004-8e1e-c54233c44cc1","Type":"ContainerStarted","Data":"5439312b40c61ed02b87a5ba0e83c9f819212342f752a4bc4ae1df735d606554"} Oct 30 18:50:13.555287 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:13.555254 5054 generic.go:356] "Generic (PLEG): container finished" podID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" containerID="5439312b40c61ed02b87a5ba0e83c9f819212342f752a4bc4ae1df735d606554" exitCode=0 Oct 30 18:50:13.555659 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:13.555323 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" event={"ID":"f1b60250-3c6a-4004-8e1e-c54233c44cc1","Type":"ContainerDied","Data":"5439312b40c61ed02b87a5ba0e83c9f819212342f752a4bc4ae1df735d606554"} Oct 30 18:50:29.644244 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:29.644205 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" event={"ID":"f1b60250-3c6a-4004-8e1e-c54233c44cc1","Type":"ContainerStarted","Data":"8f079c868859209d08109d597fcf6f540cf2ef3e47bc123bb4acc93de515724b"} Oct 30 18:50:32.660947 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:32.660913 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" event={"ID":"f1b60250-3c6a-4004-8e1e-c54233c44cc1","Type":"ContainerStarted","Data":"339f3a8db6f704f546349e8eadba12b08b2bce6f27b768804dd88bd8ffe71e0a"} Oct 30 18:50:32.661337 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:32.661038 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:50:32.852595 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:32.852545 5054 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" podStartSLOduration=3.357672142 podStartE2EDuration="33.852532837s" totalImagesPullingTime="26.595044353s" totalInitContainerRuntime="3.899816342s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:49:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:32.849782308 +0000 UTC m=+774.578161024" watchObservedRunningTime="2026-10-30 18:50:32.852532837 +0000 UTC m=+774.580911553" Oct 30 18:50:33.665303 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:33.665269 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:50:33.666797 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:33.666768 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" podUID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused" Oct 30 18:50:34.669359 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:34.669317 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" podUID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused" Oct 30 18:50:38.596894 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:50:38.596860 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:50:39.674386 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:39.674356 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:50:39.674844 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:39.674780 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" podUID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused" Oct 30 18:50:49.675752 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:49.675710 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" podUID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused" Oct 30 18:50:59.675220 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:50:59.675181 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" podUID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused" Oct 30 18:51:08.597213 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:51:08.597172 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:51:09.674929 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:51:09.674894 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" podUID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused" Oct 30 18:51:19.675211 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:51:19.675166 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" podUID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused" Oct 30 18:51:29.674988 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:51:29.674945 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" podUID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused" Oct 30 18:51:38.597396 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:51:38.597359 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:51:39.675756 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:51:39.675722 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" podUID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.35:8080: connect: connection refused" Oct 30 18:51:49.675709 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:51:49.675633 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:52:08.596991 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:52:08.596953 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:52:38.491465 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:52:38.491438 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 18:52:38.494030 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:52:38.493490 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 18:52:38.596383 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:52:38.596353 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:53:08.596644 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:53:08.596609 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:53:38.597837 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:53:38.597754 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:53:39.670687 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:39.670653 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz"] Oct 30 18:53:39.671134 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:39.670879 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" podUID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" containerName="kserve-container" containerID="cri-o://8f079c868859209d08109d597fcf6f540cf2ef3e47bc123bb4acc93de515724b" gracePeriod=30 Oct 30 18:53:39.671134 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:39.670938 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" podUID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" containerName="kube-rbac-proxy" containerID="cri-o://339f3a8db6f704f546349e8eadba12b08b2bce6f27b768804dd88bd8ffe71e0a" gracePeriod=30 Oct 30 18:53:40.408211 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:40.408178 5054 generic.go:356] "Generic (PLEG): container finished" podID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" containerID="339f3a8db6f704f546349e8eadba12b08b2bce6f27b768804dd88bd8ffe71e0a" exitCode=0 Oct 30 18:53:40.408379 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:40.408239 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" event={"ID":"f1b60250-3c6a-4004-8e1e-c54233c44cc1","Type":"ContainerDied","Data":"339f3a8db6f704f546349e8eadba12b08b2bce6f27b768804dd88bd8ffe71e0a"} Oct 30 18:53:44.111080 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.110995 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:53:44.171654 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.171627 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f1b60250-3c6a-4004-8e1e-c54233c44cc1-proxy-tls\" (UniqueName: \"kubernetes.io/secret/f1b60250-3c6a-4004-8e1e-c54233c44cc1-proxy-tls\") pod \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\" (UID: \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\") " Oct 30 18:53:44.171765 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.171671 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f1b60250-3c6a-4004-8e1e-c54233c44cc1-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f1b60250-3c6a-4004-8e1e-c54233c44cc1-kube-rbac-proxy-sar-config\") pod \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\" (UID: \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\") " Oct 30 18:53:44.171765 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.171711 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1b60250-3c6a-4004-8e1e-c54233c44cc1-kube-api-access-7jzll\" (UniqueName: \"kubernetes.io/projected/f1b60250-3c6a-4004-8e1e-c54233c44cc1-kube-api-access-7jzll\") pod \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\" (UID: \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\") " Oct 30 18:53:44.171765 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.171730 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f1b60250-3c6a-4004-8e1e-c54233c44cc1-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f1b60250-3c6a-4004-8e1e-c54233c44cc1-cabundle-cert\") pod \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\" (UID: \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\") " Oct 30 18:53:44.171765 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.171760 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f1b60250-3c6a-4004-8e1e-c54233c44cc1-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f1b60250-3c6a-4004-8e1e-c54233c44cc1-kserve-provision-location\") pod \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\" (UID: \"f1b60250-3c6a-4004-8e1e-c54233c44cc1\") " Oct 30 18:53:44.172152 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.172117 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1b60250-3c6a-4004-8e1e-c54233c44cc1-kube-rbac-proxy-sar-config" pod "f1b60250-3c6a-4004-8e1e-c54233c44cc1" (UID: "f1b60250-3c6a-4004-8e1e-c54233c44cc1"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:53:44.172152 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.172137 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1b60250-3c6a-4004-8e1e-c54233c44cc1-cabundle-cert" pod "f1b60250-3c6a-4004-8e1e-c54233c44cc1" (UID: "f1b60250-3c6a-4004-8e1e-c54233c44cc1"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:53:44.172152 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.172121 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1b60250-3c6a-4004-8e1e-c54233c44cc1-kserve-provision-location" pod "f1b60250-3c6a-4004-8e1e-c54233c44cc1" (UID: "f1b60250-3c6a-4004-8e1e-c54233c44cc1"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:53:44.173852 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.173833 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1b60250-3c6a-4004-8e1e-c54233c44cc1-kube-api-access-7jzll" pod "f1b60250-3c6a-4004-8e1e-c54233c44cc1" (UID: "f1b60250-3c6a-4004-8e1e-c54233c44cc1"). InnerVolumeSpecName "kube-api-access-7jzll". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:53:44.173926 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.173877 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1b60250-3c6a-4004-8e1e-c54233c44cc1-proxy-tls" pod "f1b60250-3c6a-4004-8e1e-c54233c44cc1" (UID: "f1b60250-3c6a-4004-8e1e-c54233c44cc1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:53:44.272620 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.272556 5054 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f1b60250-3c6a-4004-8e1e-c54233c44cc1-proxy-tls\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:53:44.272620 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.272579 5054 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f1b60250-3c6a-4004-8e1e-c54233c44cc1-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:53:44.272620 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.272588 5054 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jzll\" (UniqueName: \"kubernetes.io/projected/f1b60250-3c6a-4004-8e1e-c54233c44cc1-kube-api-access-7jzll\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:53:44.272620 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.272597 5054 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f1b60250-3c6a-4004-8e1e-c54233c44cc1-cabundle-cert\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:53:44.272620 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.272607 5054 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f1b60250-3c6a-4004-8e1e-c54233c44cc1-kserve-provision-location\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:53:44.429070 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.429041 5054 generic.go:356] "Generic (PLEG): container finished" podID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" containerID="8f079c868859209d08109d597fcf6f540cf2ef3e47bc123bb4acc93de515724b" exitCode=0 Oct 30 18:53:44.429215 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.429100 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" Oct 30 18:53:44.429215 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.429117 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" event={"ID":"f1b60250-3c6a-4004-8e1e-c54233c44cc1","Type":"ContainerDied","Data":"8f079c868859209d08109d597fcf6f540cf2ef3e47bc123bb4acc93de515724b"} Oct 30 18:53:44.429215 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.429179 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz" event={"ID":"f1b60250-3c6a-4004-8e1e-c54233c44cc1","Type":"ContainerDied","Data":"e3661f5ac11eb41b10f2901ef9f4fd227ceb8557597b41ee1c849da351dc7d8f"} Oct 30 18:53:44.429349 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.429218 5054 scope.go:122] "RemoveContainer" containerID="339f3a8db6f704f546349e8eadba12b08b2bce6f27b768804dd88bd8ffe71e0a" Oct 30 18:53:44.437280 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.437259 5054 scope.go:122] "RemoveContainer" containerID="8f079c868859209d08109d597fcf6f540cf2ef3e47bc123bb4acc93de515724b" Oct 30 18:53:44.444216 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.444198 5054 scope.go:122] "RemoveContainer" containerID="5439312b40c61ed02b87a5ba0e83c9f819212342f752a4bc4ae1df735d606554" Oct 30 18:53:44.451097 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.451080 5054 scope.go:122] "RemoveContainer" containerID="339f3a8db6f704f546349e8eadba12b08b2bce6f27b768804dd88bd8ffe71e0a" Oct 30 18:53:44.451342 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:53:44.451324 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"339f3a8db6f704f546349e8eadba12b08b2bce6f27b768804dd88bd8ffe71e0a\": container with ID starting with 339f3a8db6f704f546349e8eadba12b08b2bce6f27b768804dd88bd8ffe71e0a not found: ID does not exist" containerID="339f3a8db6f704f546349e8eadba12b08b2bce6f27b768804dd88bd8ffe71e0a" Oct 30 18:53:44.451409 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.451354 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"339f3a8db6f704f546349e8eadba12b08b2bce6f27b768804dd88bd8ffe71e0a"} err="failed to get container status \"339f3a8db6f704f546349e8eadba12b08b2bce6f27b768804dd88bd8ffe71e0a\": rpc error: code = NotFound desc = could not find container \"339f3a8db6f704f546349e8eadba12b08b2bce6f27b768804dd88bd8ffe71e0a\": container with ID starting with 339f3a8db6f704f546349e8eadba12b08b2bce6f27b768804dd88bd8ffe71e0a not found: ID does not exist" Oct 30 18:53:44.451409 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.451377 5054 scope.go:122] "RemoveContainer" containerID="8f079c868859209d08109d597fcf6f540cf2ef3e47bc123bb4acc93de515724b" Oct 30 18:53:44.451612 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:53:44.451592 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f079c868859209d08109d597fcf6f540cf2ef3e47bc123bb4acc93de515724b\": container with ID starting with 8f079c868859209d08109d597fcf6f540cf2ef3e47bc123bb4acc93de515724b not found: ID does not exist" containerID="8f079c868859209d08109d597fcf6f540cf2ef3e47bc123bb4acc93de515724b" Oct 30 18:53:44.451708 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.451614 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f079c868859209d08109d597fcf6f540cf2ef3e47bc123bb4acc93de515724b"} err="failed to get container status \"8f079c868859209d08109d597fcf6f540cf2ef3e47bc123bb4acc93de515724b\": rpc error: code = NotFound desc = could not find container \"8f079c868859209d08109d597fcf6f540cf2ef3e47bc123bb4acc93de515724b\": container with ID starting with 8f079c868859209d08109d597fcf6f540cf2ef3e47bc123bb4acc93de515724b not found: ID does not exist" Oct 30 18:53:44.451708 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.451628 5054 scope.go:122] "RemoveContainer" containerID="5439312b40c61ed02b87a5ba0e83c9f819212342f752a4bc4ae1df735d606554" Oct 30 18:53:44.451858 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:53:44.451843 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5439312b40c61ed02b87a5ba0e83c9f819212342f752a4bc4ae1df735d606554\": container with ID starting with 5439312b40c61ed02b87a5ba0e83c9f819212342f752a4bc4ae1df735d606554 not found: ID does not exist" containerID="5439312b40c61ed02b87a5ba0e83c9f819212342f752a4bc4ae1df735d606554" Oct 30 18:53:44.451898 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.451861 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5439312b40c61ed02b87a5ba0e83c9f819212342f752a4bc4ae1df735d606554"} err="failed to get container status \"5439312b40c61ed02b87a5ba0e83c9f819212342f752a4bc4ae1df735d606554\": rpc error: code = NotFound desc = could not find container \"5439312b40c61ed02b87a5ba0e83c9f819212342f752a4bc4ae1df735d606554\": container with ID starting with 5439312b40c61ed02b87a5ba0e83c9f819212342f752a4bc4ae1df735d606554 not found: ID does not exist" Oct 30 18:53:44.595667 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.595595 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz"] Oct 30 18:53:44.602381 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:44.602356 5054 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-rollback-predictor-554c7cc56d-j7cqz"] Oct 30 18:53:46.421054 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:53:46.421024 5054 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" path="/var/lib/kubelet/pods/f1b60250-3c6a-4004-8e1e-c54233c44cc1/volumes" Oct 30 18:54:08.596673 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:54:08.596633 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:54:17.529573 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.529539 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7"] Oct 30 18:54:17.530079 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.529832 5054 cpu_manager.go:414] "containerMap: removing container" podUID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" containerName="storage-initializer" Oct 30 18:54:17.530079 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.529842 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" containerName="storage-initializer" Oct 30 18:54:17.530079 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.529853 5054 cpu_manager.go:414] "containerMap: removing container" podUID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" containerName="kube-rbac-proxy" Oct 30 18:54:17.530079 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.529858 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" containerName="kube-rbac-proxy" Oct 30 18:54:17.530079 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.529885 5054 cpu_manager.go:414] "containerMap: removing container" podUID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" containerName="kserve-container" Oct 30 18:54:17.530079 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.529891 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" containerName="kserve-container" Oct 30 18:54:17.530079 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.529947 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" containerName="kserve-container" Oct 30 18:54:17.530079 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.529956 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="f1b60250-3c6a-4004-8e1e-c54233c44cc1" containerName="kube-rbac-proxy" Oct 30 18:54:17.533251 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.533231 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:54:17.586407 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.586375 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-1fd37956ac\"/\"seaweedfs-s3-creds\"" Oct 30 18:54:17.631519 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.631487 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7a11c704-8ba8-4d32-998e-abbab48c0667-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7\" (UID: \"7a11c704-8ba8-4d32-998e-abbab48c0667\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:54:17.631690 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.631533 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d592j\" (UniqueName: \"kubernetes.io/projected/7a11c704-8ba8-4d32-998e-abbab48c0667-kube-api-access-d592j\") pod \"isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7\" (UID: \"7a11c704-8ba8-4d32-998e-abbab48c0667\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:54:17.631690 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.631599 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7a11c704-8ba8-4d32-998e-abbab48c0667-cabundle-cert\") pod \"isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7\" (UID: \"7a11c704-8ba8-4d32-998e-abbab48c0667\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:54:17.631690 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.631646 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7a11c704-8ba8-4d32-998e-abbab48c0667-proxy-tls\") pod \"isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7\" (UID: \"7a11c704-8ba8-4d32-998e-abbab48c0667\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:54:17.631805 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.631723 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7a11c704-8ba8-4d32-998e-abbab48c0667-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7\" (UID: \"7a11c704-8ba8-4d32-998e-abbab48c0667\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:54:17.633581 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.633558 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1fd37956ac\"/\"isvc-xgboost-graph-raw-bdcff-kube-rbac-proxy-sar-config\"" Oct 30 18:54:17.633690 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.633583 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-1fd37956ac\"/\"default-dockercfg-tx9z6\"" Oct 30 18:54:17.633690 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.633584 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1fd37956ac\"/\"kube-root-ca.crt\"" Oct 30 18:54:17.633690 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.633595 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1fd37956ac\"/\"odh-kserve-custom-ca-bundle\"" Oct 30 18:54:17.633690 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.633648 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-1fd37956ac\"/\"isvc-xgboost-graph-raw-bdcff-predictor-serving-cert\"" Oct 30 18:54:17.633819 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.633740 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1fd37956ac\"/\"openshift-service-ca.crt\"" Oct 30 18:54:17.635402 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.635384 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7"] Oct 30 18:54:17.732979 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.732949 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7a11c704-8ba8-4d32-998e-abbab48c0667-cabundle-cert\") pod \"isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7\" (UID: \"7a11c704-8ba8-4d32-998e-abbab48c0667\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:54:17.733154 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.732993 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7a11c704-8ba8-4d32-998e-abbab48c0667-proxy-tls\") pod \"isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7\" (UID: \"7a11c704-8ba8-4d32-998e-abbab48c0667\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:54:17.733154 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.733037 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7a11c704-8ba8-4d32-998e-abbab48c0667-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7\" (UID: \"7a11c704-8ba8-4d32-998e-abbab48c0667\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:54:17.733154 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.733087 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7a11c704-8ba8-4d32-998e-abbab48c0667-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7\" (UID: \"7a11c704-8ba8-4d32-998e-abbab48c0667\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:54:17.733154 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.733112 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d592j\" (UniqueName: \"kubernetes.io/projected/7a11c704-8ba8-4d32-998e-abbab48c0667-kube-api-access-d592j\") pod \"isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7\" (UID: \"7a11c704-8ba8-4d32-998e-abbab48c0667\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:54:17.733543 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.733512 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7a11c704-8ba8-4d32-998e-abbab48c0667-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7\" (UID: \"7a11c704-8ba8-4d32-998e-abbab48c0667\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:54:17.733696 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.733673 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7a11c704-8ba8-4d32-998e-abbab48c0667-cabundle-cert\") pod \"isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7\" (UID: \"7a11c704-8ba8-4d32-998e-abbab48c0667\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:54:17.733774 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.733744 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7a11c704-8ba8-4d32-998e-abbab48c0667-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7\" (UID: \"7a11c704-8ba8-4d32-998e-abbab48c0667\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:54:17.735582 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.735561 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7a11c704-8ba8-4d32-998e-abbab48c0667-proxy-tls\") pod \"isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7\" (UID: \"7a11c704-8ba8-4d32-998e-abbab48c0667\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:54:17.794040 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.793966 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d592j\" (UniqueName: \"kubernetes.io/projected/7a11c704-8ba8-4d32-998e-abbab48c0667-kube-api-access-d592j\") pod \"isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7\" (UID: \"7a11c704-8ba8-4d32-998e-abbab48c0667\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:54:17.849557 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:17.849536 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:54:18.268381 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:18.268337 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7"] Oct 30 18:54:18.274470 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:54:18.274436 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a11c704_8ba8_4d32_998e_abbab48c0667.slice/crio-02a04bffa563dc6521e3b1f4bedb319cb9690215d6b20cfdba81ac085dfc9eaa WatchSource:0}: Error finding container 02a04bffa563dc6521e3b1f4bedb319cb9690215d6b20cfdba81ac085dfc9eaa: Status 404 returned error can't find the container with id 02a04bffa563dc6521e3b1f4bedb319cb9690215d6b20cfdba81ac085dfc9eaa Oct 30 18:54:18.570550 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:18.570465 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" event={"ID":"7a11c704-8ba8-4d32-998e-abbab48c0667","Type":"ContainerStarted","Data":"02d606431789ae5218d8ee7a09f207d252d48c4420eccd3c1ef611db3fe56367"} Oct 30 18:54:18.570550 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:18.570527 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" event={"ID":"7a11c704-8ba8-4d32-998e-abbab48c0667","Type":"ContainerStarted","Data":"02a04bffa563dc6521e3b1f4bedb319cb9690215d6b20cfdba81ac085dfc9eaa"} Oct 30 18:54:22.587121 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:22.587033 5054 generic.go:356] "Generic (PLEG): container finished" podID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerID="02d606431789ae5218d8ee7a09f207d252d48c4420eccd3c1ef611db3fe56367" exitCode=0 Oct 30 18:54:22.587121 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:22.587107 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" event={"ID":"7a11c704-8ba8-4d32-998e-abbab48c0667","Type":"ContainerDied","Data":"02d606431789ae5218d8ee7a09f207d252d48c4420eccd3c1ef611db3fe56367"} Oct 30 18:54:38.599586 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:54:38.599539 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:54:41.676875 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:41.676844 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" event={"ID":"7a11c704-8ba8-4d32-998e-abbab48c0667","Type":"ContainerStarted","Data":"082f2de40067ed50456c693cbfc01fa5534053dca3f0afce732c9c3645712fb1"} Oct 30 18:54:42.682953 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:42.682919 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" event={"ID":"7a11c704-8ba8-4d32-998e-abbab48c0667","Type":"ContainerStarted","Data":"d846e573fa09e9c492bf702b710735b58df5ea08f84258a1ef61309ef3270078"} Oct 30 18:54:42.683316 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:42.683136 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:54:42.850435 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:42.850387 5054 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" podStartSLOduration=3.017958703 podStartE2EDuration="25.850374663s" totalImagesPullingTime="18.978040932s" totalInitContainerRuntime="3.854375028s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:42.84909436 +0000 UTC m=+1024.577473075" watchObservedRunningTime="2026-10-30 18:54:42.850374663 +0000 UTC m=+1024.578753379" Oct 30 18:54:43.687333 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:43.687298 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:54:43.688394 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:43.688355 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" podUID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused" Oct 30 18:54:44.691645 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:44.691600 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" podUID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused" Oct 30 18:54:49.697082 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:49.696991 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:54:49.697618 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:49.697591 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" podUID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused" Oct 30 18:54:59.698390 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:54:59.698355 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" podUID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused" Oct 30 18:55:08.596834 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:55:08.596780 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:55:09.698210 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:09.698163 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" podUID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused" Oct 30 18:55:19.698405 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:19.698365 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" podUID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused" Oct 30 18:55:26.116554 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:26.116520 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-6x2g6-kxhlf"] Oct 30 18:55:26.119840 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:26.119823 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-6x2g6-kxhlf" Oct 30 18:55:26.175899 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:26.175869 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-6x2g6-kxhlf"] Oct 30 18:55:26.185830 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:26.185800 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/647f31b7-3d07-41ac-925f-6585e666a757-archives-path\") pod \"periodic-gathering-6x2g6-kxhlf\" (UID: \"647f31b7-3d07-41ac-925f-6585e666a757\") " pod="openshift-insights/periodic-gathering-6x2g6-kxhlf" Oct 30 18:55:26.185830 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:26.185829 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/647f31b7-3d07-41ac-925f-6585e666a757-trusted-ca-bundle\") pod \"periodic-gathering-6x2g6-kxhlf\" (UID: \"647f31b7-3d07-41ac-925f-6585e666a757\") " pod="openshift-insights/periodic-gathering-6x2g6-kxhlf" Oct 30 18:55:26.186059 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:26.185852 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/647f31b7-3d07-41ac-925f-6585e666a757-service-ca-bundle\") pod \"periodic-gathering-6x2g6-kxhlf\" (UID: \"647f31b7-3d07-41ac-925f-6585e666a757\") " pod="openshift-insights/periodic-gathering-6x2g6-kxhlf" Oct 30 18:55:26.186059 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:26.185996 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gphc\" (UniqueName: \"kubernetes.io/projected/647f31b7-3d07-41ac-925f-6585e666a757-kube-api-access-4gphc\") pod \"periodic-gathering-6x2g6-kxhlf\" (UID: \"647f31b7-3d07-41ac-925f-6585e666a757\") " pod="openshift-insights/periodic-gathering-6x2g6-kxhlf" Oct 30 18:55:26.286990 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:26.286961 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gphc\" (UniqueName: \"kubernetes.io/projected/647f31b7-3d07-41ac-925f-6585e666a757-kube-api-access-4gphc\") pod \"periodic-gathering-6x2g6-kxhlf\" (UID: \"647f31b7-3d07-41ac-925f-6585e666a757\") " pod="openshift-insights/periodic-gathering-6x2g6-kxhlf" Oct 30 18:55:26.287146 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:26.287045 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/647f31b7-3d07-41ac-925f-6585e666a757-archives-path\") pod \"periodic-gathering-6x2g6-kxhlf\" (UID: \"647f31b7-3d07-41ac-925f-6585e666a757\") " pod="openshift-insights/periodic-gathering-6x2g6-kxhlf" Oct 30 18:55:26.287146 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:26.287061 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/647f31b7-3d07-41ac-925f-6585e666a757-trusted-ca-bundle\") pod \"periodic-gathering-6x2g6-kxhlf\" (UID: \"647f31b7-3d07-41ac-925f-6585e666a757\") " pod="openshift-insights/periodic-gathering-6x2g6-kxhlf" Oct 30 18:55:26.287146 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:26.287088 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/647f31b7-3d07-41ac-925f-6585e666a757-service-ca-bundle\") pod \"periodic-gathering-6x2g6-kxhlf\" (UID: \"647f31b7-3d07-41ac-925f-6585e666a757\") " pod="openshift-insights/periodic-gathering-6x2g6-kxhlf" Oct 30 18:55:26.287471 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:26.287452 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/647f31b7-3d07-41ac-925f-6585e666a757-archives-path\") pod \"periodic-gathering-6x2g6-kxhlf\" (UID: \"647f31b7-3d07-41ac-925f-6585e666a757\") " pod="openshift-insights/periodic-gathering-6x2g6-kxhlf" Oct 30 18:55:26.287717 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:26.287701 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/647f31b7-3d07-41ac-925f-6585e666a757-service-ca-bundle\") pod \"periodic-gathering-6x2g6-kxhlf\" (UID: \"647f31b7-3d07-41ac-925f-6585e666a757\") " pod="openshift-insights/periodic-gathering-6x2g6-kxhlf" Oct 30 18:55:26.287829 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:26.287813 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/647f31b7-3d07-41ac-925f-6585e666a757-trusted-ca-bundle\") pod \"periodic-gathering-6x2g6-kxhlf\" (UID: \"647f31b7-3d07-41ac-925f-6585e666a757\") " pod="openshift-insights/periodic-gathering-6x2g6-kxhlf" Oct 30 18:55:26.348339 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:26.348309 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gphc\" (UniqueName: \"kubernetes.io/projected/647f31b7-3d07-41ac-925f-6585e666a757-kube-api-access-4gphc\") pod \"periodic-gathering-6x2g6-kxhlf\" (UID: \"647f31b7-3d07-41ac-925f-6585e666a757\") " pod="openshift-insights/periodic-gathering-6x2g6-kxhlf" Oct 30 18:55:26.431154 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:26.431085 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-6x2g6-kxhlf" Oct 30 18:55:26.805304 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:26.805224 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-6x2g6-kxhlf"] Oct 30 18:55:26.811386 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:55:26.811358 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod647f31b7_3d07_41ac_925f_6585e666a757.slice/crio-4fc5172851c9c589f883d227c290ca3fd0cabfc3a295c250042c27f2675f910f WatchSource:0}: Error finding container 4fc5172851c9c589f883d227c290ca3fd0cabfc3a295c250042c27f2675f910f: Status 404 returned error can't find the container with id 4fc5172851c9c589f883d227c290ca3fd0cabfc3a295c250042c27f2675f910f Oct 30 18:55:26.868189 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:26.868156 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-6x2g6-kxhlf" event={"ID":"647f31b7-3d07-41ac-925f-6585e666a757","Type":"ContainerStarted","Data":"25aa7893d56712957a0604fb4c3a24277bc3c65751dbc4a8a05405375dd2d62e"} Oct 30 18:55:26.868296 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:26.868198 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-6x2g6-kxhlf" event={"ID":"647f31b7-3d07-41ac-925f-6585e666a757","Type":"ContainerStarted","Data":"4fc5172851c9c589f883d227c290ca3fd0cabfc3a295c250042c27f2675f910f"} Oct 30 18:55:29.697840 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:29.697801 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" podUID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused" Oct 30 18:55:38.597424 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:55:38.597386 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:55:39.698221 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:39.698185 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" podUID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused" Oct 30 18:55:49.699051 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:49.699022 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:55:49.914252 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:55:49.914190 5054 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-6x2g6-kxhlf" podStartSLOduration=23.914174317 podStartE2EDuration="23.914174317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:26.981846791 +0000 UTC m=+1068.710225506" watchObservedRunningTime="2026-10-30 18:55:49.914174317 +0000 UTC m=+1091.642553075" Oct 30 18:56:07.736628 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:07.736596 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7"] Oct 30 18:56:07.737174 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:07.736858 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" podUID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerName="kserve-container" containerID="cri-o://082f2de40067ed50456c693cbfc01fa5534053dca3f0afce732c9c3645712fb1" gracePeriod=30 Oct 30 18:56:07.737174 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:07.736931 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" podUID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerName="kube-rbac-proxy" containerID="cri-o://d846e573fa09e9c492bf702b710735b58df5ea08f84258a1ef61309ef3270078" gracePeriod=30 Oct 30 18:56:08.040709 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:08.040629 5054 generic.go:356] "Generic (PLEG): container finished" podID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerID="d846e573fa09e9c492bf702b710735b58df5ea08f84258a1ef61309ef3270078" exitCode=0 Oct 30 18:56:08.040845 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:08.040702 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" event={"ID":"7a11c704-8ba8-4d32-998e-abbab48c0667","Type":"ContainerDied","Data":"d846e573fa09e9c492bf702b710735b58df5ea08f84258a1ef61309ef3270078"} Oct 30 18:56:08.596350 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:56:08.596314 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:56:09.692335 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:09.692292 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" podUID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.36:8643/healthz\": dial tcp 10.132.0.36:8643: connect: connection refused" Oct 30 18:56:09.697717 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:09.697686 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" podUID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.36:8080: connect: connection refused" Oct 30 18:56:11.480333 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:11.480309 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:56:11.577252 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:11.577182 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7a11c704-8ba8-4d32-998e-abbab48c0667-proxy-tls\" (UniqueName: \"kubernetes.io/secret/7a11c704-8ba8-4d32-998e-abbab48c0667-proxy-tls\") pod \"7a11c704-8ba8-4d32-998e-abbab48c0667\" (UID: \"7a11c704-8ba8-4d32-998e-abbab48c0667\") " Oct 30 18:56:11.577369 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:11.577278 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7a11c704-8ba8-4d32-998e-abbab48c0667-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7a11c704-8ba8-4d32-998e-abbab48c0667-cabundle-cert\") pod \"7a11c704-8ba8-4d32-998e-abbab48c0667\" (UID: \"7a11c704-8ba8-4d32-998e-abbab48c0667\") " Oct 30 18:56:11.577369 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:11.577326 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7a11c704-8ba8-4d32-998e-abbab48c0667-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7a11c704-8ba8-4d32-998e-abbab48c0667-kube-rbac-proxy-sar-config\") pod \"7a11c704-8ba8-4d32-998e-abbab48c0667\" (UID: \"7a11c704-8ba8-4d32-998e-abbab48c0667\") " Oct 30 18:56:11.577369 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:11.577359 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a11c704-8ba8-4d32-998e-abbab48c0667-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7a11c704-8ba8-4d32-998e-abbab48c0667-kserve-provision-location\") pod \"7a11c704-8ba8-4d32-998e-abbab48c0667\" (UID: \"7a11c704-8ba8-4d32-998e-abbab48c0667\") " Oct 30 18:56:11.577496 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:11.577393 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7a11c704-8ba8-4d32-998e-abbab48c0667-kube-api-access-d592j\" (UniqueName: \"kubernetes.io/projected/7a11c704-8ba8-4d32-998e-abbab48c0667-kube-api-access-d592j\") pod \"7a11c704-8ba8-4d32-998e-abbab48c0667\" (UID: \"7a11c704-8ba8-4d32-998e-abbab48c0667\") " Oct 30 18:56:11.577717 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:11.577658 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a11c704-8ba8-4d32-998e-abbab48c0667-kserve-provision-location" pod "7a11c704-8ba8-4d32-998e-abbab48c0667" (UID: "7a11c704-8ba8-4d32-998e-abbab48c0667"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:56:11.577717 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:11.577696 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a11c704-8ba8-4d32-998e-abbab48c0667-kube-rbac-proxy-sar-config" pod "7a11c704-8ba8-4d32-998e-abbab48c0667" (UID: "7a11c704-8ba8-4d32-998e-abbab48c0667"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:56:11.577717 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:11.577707 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a11c704-8ba8-4d32-998e-abbab48c0667-cabundle-cert" pod "7a11c704-8ba8-4d32-998e-abbab48c0667" (UID: "7a11c704-8ba8-4d32-998e-abbab48c0667"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:56:11.578065 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:11.577940 5054 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7a11c704-8ba8-4d32-998e-abbab48c0667-cabundle-cert\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:56:11.578065 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:11.577961 5054 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7a11c704-8ba8-4d32-998e-abbab48c0667-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:56:11.578065 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:11.577992 5054 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7a11c704-8ba8-4d32-998e-abbab48c0667-kserve-provision-location\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:56:11.579482 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:11.579459 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a11c704-8ba8-4d32-998e-abbab48c0667-proxy-tls" pod "7a11c704-8ba8-4d32-998e-abbab48c0667" (UID: "7a11c704-8ba8-4d32-998e-abbab48c0667"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:56:11.579612 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:11.579590 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a11c704-8ba8-4d32-998e-abbab48c0667-kube-api-access-d592j" pod "7a11c704-8ba8-4d32-998e-abbab48c0667" (UID: "7a11c704-8ba8-4d32-998e-abbab48c0667"). InnerVolumeSpecName "kube-api-access-d592j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:56:11.679160 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:11.679131 5054 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d592j\" (UniqueName: \"kubernetes.io/projected/7a11c704-8ba8-4d32-998e-abbab48c0667-kube-api-access-d592j\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:56:11.679160 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:11.679158 5054 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7a11c704-8ba8-4d32-998e-abbab48c0667-proxy-tls\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:56:12.058601 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:12.058567 5054 generic.go:356] "Generic (PLEG): container finished" podID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerID="082f2de40067ed50456c693cbfc01fa5534053dca3f0afce732c9c3645712fb1" exitCode=0 Oct 30 18:56:12.058804 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:12.058638 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" Oct 30 18:56:12.058804 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:12.058655 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" event={"ID":"7a11c704-8ba8-4d32-998e-abbab48c0667","Type":"ContainerDied","Data":"082f2de40067ed50456c693cbfc01fa5534053dca3f0afce732c9c3645712fb1"} Oct 30 18:56:12.058804 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:12.058712 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7" event={"ID":"7a11c704-8ba8-4d32-998e-abbab48c0667","Type":"ContainerDied","Data":"02a04bffa563dc6521e3b1f4bedb319cb9690215d6b20cfdba81ac085dfc9eaa"} Oct 30 18:56:12.058804 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:12.058739 5054 scope.go:122] "RemoveContainer" containerID="d846e573fa09e9c492bf702b710735b58df5ea08f84258a1ef61309ef3270078" Oct 30 18:56:12.067219 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:12.067189 5054 scope.go:122] "RemoveContainer" containerID="082f2de40067ed50456c693cbfc01fa5534053dca3f0afce732c9c3645712fb1" Oct 30 18:56:12.074229 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:12.074206 5054 scope.go:122] "RemoveContainer" containerID="02d606431789ae5218d8ee7a09f207d252d48c4420eccd3c1ef611db3fe56367" Oct 30 18:56:12.081299 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:12.081281 5054 scope.go:122] "RemoveContainer" containerID="d846e573fa09e9c492bf702b710735b58df5ea08f84258a1ef61309ef3270078" Oct 30 18:56:12.081560 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:56:12.081542 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d846e573fa09e9c492bf702b710735b58df5ea08f84258a1ef61309ef3270078\": container with ID starting with d846e573fa09e9c492bf702b710735b58df5ea08f84258a1ef61309ef3270078 not found: ID does not exist" containerID="d846e573fa09e9c492bf702b710735b58df5ea08f84258a1ef61309ef3270078" Oct 30 18:56:12.081601 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:12.081570 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d846e573fa09e9c492bf702b710735b58df5ea08f84258a1ef61309ef3270078"} err="failed to get container status \"d846e573fa09e9c492bf702b710735b58df5ea08f84258a1ef61309ef3270078\": rpc error: code = NotFound desc = could not find container \"d846e573fa09e9c492bf702b710735b58df5ea08f84258a1ef61309ef3270078\": container with ID starting with d846e573fa09e9c492bf702b710735b58df5ea08f84258a1ef61309ef3270078 not found: ID does not exist" Oct 30 18:56:12.081601 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:12.081587 5054 scope.go:122] "RemoveContainer" containerID="082f2de40067ed50456c693cbfc01fa5534053dca3f0afce732c9c3645712fb1" Oct 30 18:56:12.081835 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:56:12.081817 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"082f2de40067ed50456c693cbfc01fa5534053dca3f0afce732c9c3645712fb1\": container with ID starting with 082f2de40067ed50456c693cbfc01fa5534053dca3f0afce732c9c3645712fb1 not found: ID does not exist" containerID="082f2de40067ed50456c693cbfc01fa5534053dca3f0afce732c9c3645712fb1" Oct 30 18:56:12.081883 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:12.081839 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"082f2de40067ed50456c693cbfc01fa5534053dca3f0afce732c9c3645712fb1"} err="failed to get container status \"082f2de40067ed50456c693cbfc01fa5534053dca3f0afce732c9c3645712fb1\": rpc error: code = NotFound desc = could not find container \"082f2de40067ed50456c693cbfc01fa5534053dca3f0afce732c9c3645712fb1\": container with ID starting with 082f2de40067ed50456c693cbfc01fa5534053dca3f0afce732c9c3645712fb1 not found: ID does not exist" Oct 30 18:56:12.081883 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:12.081854 5054 scope.go:122] "RemoveContainer" containerID="02d606431789ae5218d8ee7a09f207d252d48c4420eccd3c1ef611db3fe56367" Oct 30 18:56:12.082092 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:56:12.082075 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02d606431789ae5218d8ee7a09f207d252d48c4420eccd3c1ef611db3fe56367\": container with ID starting with 02d606431789ae5218d8ee7a09f207d252d48c4420eccd3c1ef611db3fe56367 not found: ID does not exist" containerID="02d606431789ae5218d8ee7a09f207d252d48c4420eccd3c1ef611db3fe56367" Oct 30 18:56:12.082135 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:12.082099 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02d606431789ae5218d8ee7a09f207d252d48c4420eccd3c1ef611db3fe56367"} err="failed to get container status \"02d606431789ae5218d8ee7a09f207d252d48c4420eccd3c1ef611db3fe56367\": rpc error: code = NotFound desc = could not find container \"02d606431789ae5218d8ee7a09f207d252d48c4420eccd3c1ef611db3fe56367\": container with ID starting with 02d606431789ae5218d8ee7a09f207d252d48c4420eccd3c1ef611db3fe56367 not found: ID does not exist" Oct 30 18:56:12.223495 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:12.223461 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7"] Oct 30 18:56:12.229672 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:12.229645 5054 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-bdcff-predictor-76c876ddbd-5d9s7"] Oct 30 18:56:12.420933 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:12.420893 5054 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7a11c704-8ba8-4d32-998e-abbab48c0667" path="/var/lib/kubelet/pods/7a11c704-8ba8-4d32-998e-abbab48c0667/volumes" Oct 30 18:56:17.297021 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.296984 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq"] Oct 30 18:56:17.297398 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.297324 5054 cpu_manager.go:414] "containerMap: removing container" podUID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerName="kube-rbac-proxy" Oct 30 18:56:17.297398 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.297335 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerName="kube-rbac-proxy" Oct 30 18:56:17.297398 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.297349 5054 cpu_manager.go:414] "containerMap: removing container" podUID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerName="kserve-container" Oct 30 18:56:17.297398 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.297354 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerName="kserve-container" Oct 30 18:56:17.297398 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.297370 5054 cpu_manager.go:414] "containerMap: removing container" podUID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerName="storage-initializer" Oct 30 18:56:17.297398 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.297375 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerName="storage-initializer" Oct 30 18:56:17.297575 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.297420 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerName="kube-rbac-proxy" Oct 30 18:56:17.297575 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.297428 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="7a11c704-8ba8-4d32-998e-abbab48c0667" containerName="kserve-container" Oct 30 18:56:17.302167 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.302149 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:56:17.357010 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.356987 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-1fd37956ac\"/\"isvc-xgboost-graph-raw-hpa-6d75d-predictor-serving-cert\"" Oct 30 18:56:17.357129 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.357056 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-1fd37956ac\"/\"seaweedfs-s3-creds\"" Oct 30 18:56:17.402736 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.402702 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1fd37956ac\"/\"openshift-service-ca.crt\"" Oct 30 18:56:17.402736 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.402730 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1fd37956ac\"/\"kube-root-ca.crt\"" Oct 30 18:56:17.402928 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.402706 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1fd37956ac\"/\"isvc-xgboost-graph-raw-hpa-6d75d-kube-rbac-proxy-sar-config\"" Oct 30 18:56:17.402928 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.402706 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw0-1fd37956ac\"/\"odh-kserve-custom-ca-bundle\"" Oct 30 18:56:17.402996 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.402928 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw0-1fd37956ac\"/\"default-dockercfg-tx9z6\"" Oct 30 18:56:17.404119 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.404101 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq"] Oct 30 18:56:17.424320 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.424288 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bc6b5829-a6bc-4764-8051-63ef89fa9287-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq\" (UID: \"bc6b5829-a6bc-4764-8051-63ef89fa9287\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:56:17.424618 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.424345 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bc6b5829-a6bc-4764-8051-63ef89fa9287-proxy-tls\") pod \"isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq\" (UID: \"bc6b5829-a6bc-4764-8051-63ef89fa9287\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:56:17.424618 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.424474 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gqwd\" (UniqueName: \"kubernetes.io/projected/bc6b5829-a6bc-4764-8051-63ef89fa9287-kube-api-access-6gqwd\") pod \"isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq\" (UID: \"bc6b5829-a6bc-4764-8051-63ef89fa9287\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:56:17.424618 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.424501 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bc6b5829-a6bc-4764-8051-63ef89fa9287-cabundle-cert\") pod \"isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq\" (UID: \"bc6b5829-a6bc-4764-8051-63ef89fa9287\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:56:17.424618 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.424580 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bc6b5829-a6bc-4764-8051-63ef89fa9287-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq\" (UID: \"bc6b5829-a6bc-4764-8051-63ef89fa9287\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:56:17.525410 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.525381 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bc6b5829-a6bc-4764-8051-63ef89fa9287-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq\" (UID: \"bc6b5829-a6bc-4764-8051-63ef89fa9287\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:56:17.525561 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.525414 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bc6b5829-a6bc-4764-8051-63ef89fa9287-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq\" (UID: \"bc6b5829-a6bc-4764-8051-63ef89fa9287\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:56:17.525561 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.525457 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bc6b5829-a6bc-4764-8051-63ef89fa9287-proxy-tls\") pod \"isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq\" (UID: \"bc6b5829-a6bc-4764-8051-63ef89fa9287\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:56:17.525561 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.525495 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gqwd\" (UniqueName: \"kubernetes.io/projected/bc6b5829-a6bc-4764-8051-63ef89fa9287-kube-api-access-6gqwd\") pod \"isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq\" (UID: \"bc6b5829-a6bc-4764-8051-63ef89fa9287\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:56:17.525561 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.525512 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bc6b5829-a6bc-4764-8051-63ef89fa9287-cabundle-cert\") pod \"isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq\" (UID: \"bc6b5829-a6bc-4764-8051-63ef89fa9287\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:56:17.525851 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.525822 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bc6b5829-a6bc-4764-8051-63ef89fa9287-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq\" (UID: \"bc6b5829-a6bc-4764-8051-63ef89fa9287\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:56:17.526105 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.526084 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bc6b5829-a6bc-4764-8051-63ef89fa9287-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq\" (UID: \"bc6b5829-a6bc-4764-8051-63ef89fa9287\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:56:17.526191 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.526173 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bc6b5829-a6bc-4764-8051-63ef89fa9287-cabundle-cert\") pod \"isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq\" (UID: \"bc6b5829-a6bc-4764-8051-63ef89fa9287\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:56:17.527882 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.527859 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bc6b5829-a6bc-4764-8051-63ef89fa9287-proxy-tls\") pod \"isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq\" (UID: \"bc6b5829-a6bc-4764-8051-63ef89fa9287\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:56:17.587170 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.587121 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gqwd\" (UniqueName: \"kubernetes.io/projected/bc6b5829-a6bc-4764-8051-63ef89fa9287-kube-api-access-6gqwd\") pod \"isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq\" (UID: \"bc6b5829-a6bc-4764-8051-63ef89fa9287\") " pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:56:17.617869 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:17.617846 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:56:18.051602 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:18.051568 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq"] Oct 30 18:56:18.057185 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:56:18.057159 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc6b5829_a6bc_4764_8051_63ef89fa9287.slice/crio-3adf2e769e6ccaed4179f30504c121ad6af58123111154c12a995c154ecf328f WatchSource:0}: Error finding container 3adf2e769e6ccaed4179f30504c121ad6af58123111154c12a995c154ecf328f: Status 404 returned error can't find the container with id 3adf2e769e6ccaed4179f30504c121ad6af58123111154c12a995c154ecf328f Oct 30 18:56:18.085056 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:18.085024 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" event={"ID":"bc6b5829-a6bc-4764-8051-63ef89fa9287","Type":"ContainerStarted","Data":"3adf2e769e6ccaed4179f30504c121ad6af58123111154c12a995c154ecf328f"} Oct 30 18:56:19.091024 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:19.090916 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" event={"ID":"bc6b5829-a6bc-4764-8051-63ef89fa9287","Type":"ContainerStarted","Data":"9234af7a89559eda38b98265acaeb6103a649edbc05616f49754565538a038f7"} Oct 30 18:56:23.107366 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:23.107331 5054 generic.go:356] "Generic (PLEG): container finished" podID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerID="9234af7a89559eda38b98265acaeb6103a649edbc05616f49754565538a038f7" exitCode=0 Oct 30 18:56:23.107797 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:23.107403 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" event={"ID":"bc6b5829-a6bc-4764-8051-63ef89fa9287","Type":"ContainerDied","Data":"9234af7a89559eda38b98265acaeb6103a649edbc05616f49754565538a038f7"} Oct 30 18:56:24.113817 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:24.113784 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" event={"ID":"bc6b5829-a6bc-4764-8051-63ef89fa9287","Type":"ContainerStarted","Data":"3fb53fc7d7fbf2373cca41b8ae84302dd3024ff86647af75180c7fff9831833a"} Oct 30 18:56:24.114219 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:24.113842 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" event={"ID":"bc6b5829-a6bc-4764-8051-63ef89fa9287","Type":"ContainerStarted","Data":"b91b0ecbb34a44f38c5c4f3f187c17bf59009740ebe3c4475147aafe30336850"} Oct 30 18:56:24.114219 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:24.114170 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:56:24.114321 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:24.114301 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:56:24.115606 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:24.115578 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" podUID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused" Oct 30 18:56:24.233790 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:24.233744 5054 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" podStartSLOduration=3.099218705 podStartE2EDuration="7.233732272s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.134513567s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:24.231968333 +0000 UTC m=+1125.960347063" watchObservedRunningTime="2026-10-30 18:56:24.233732272 +0000 UTC m=+1125.962111040" Oct 30 18:56:25.117480 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:25.117444 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" podUID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused" Oct 30 18:56:26.121387 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:26.121350 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" podUID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused" Oct 30 18:56:31.125599 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:31.125570 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:56:31.126069 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:31.126040 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" podUID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused" Oct 30 18:56:38.597557 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:56:38.597524 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:56:41.126220 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:41.126176 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" podUID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused" Oct 30 18:56:51.126289 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:51.126249 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" podUID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused" Oct 30 18:56:57.246536 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:57.246496 5054 generic.go:356] "Generic (PLEG): container finished" podID="647f31b7-3d07-41ac-925f-6585e666a757" containerID="25aa7893d56712957a0604fb4c3a24277bc3c65751dbc4a8a05405375dd2d62e" exitCode=0 Oct 30 18:56:57.246903 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:57.246567 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-6x2g6-kxhlf" event={"ID":"647f31b7-3d07-41ac-925f-6585e666a757","Type":"ContainerDied","Data":"25aa7893d56712957a0604fb4c3a24277bc3c65751dbc4a8a05405375dd2d62e"} Oct 30 18:56:58.382343 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:58.382318 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-6x2g6-kxhlf" Oct 30 18:56:58.447112 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:58.447089 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/647f31b7-3d07-41ac-925f-6585e666a757-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/647f31b7-3d07-41ac-925f-6585e666a757-service-ca-bundle\") pod \"647f31b7-3d07-41ac-925f-6585e666a757\" (UID: \"647f31b7-3d07-41ac-925f-6585e666a757\") " Oct 30 18:56:58.447262 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:58.447193 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/647f31b7-3d07-41ac-925f-6585e666a757-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/647f31b7-3d07-41ac-925f-6585e666a757-archives-path\") pod \"647f31b7-3d07-41ac-925f-6585e666a757\" (UID: \"647f31b7-3d07-41ac-925f-6585e666a757\") " Oct 30 18:56:58.447262 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:58.447221 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/647f31b7-3d07-41ac-925f-6585e666a757-kube-api-access-4gphc\" (UniqueName: \"kubernetes.io/projected/647f31b7-3d07-41ac-925f-6585e666a757-kube-api-access-4gphc\") pod \"647f31b7-3d07-41ac-925f-6585e666a757\" (UID: \"647f31b7-3d07-41ac-925f-6585e666a757\") " Oct 30 18:56:58.447262 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:58.447257 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/647f31b7-3d07-41ac-925f-6585e666a757-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/647f31b7-3d07-41ac-925f-6585e666a757-trusted-ca-bundle\") pod \"647f31b7-3d07-41ac-925f-6585e666a757\" (UID: \"647f31b7-3d07-41ac-925f-6585e666a757\") " Oct 30 18:56:58.447562 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:58.447538 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/647f31b7-3d07-41ac-925f-6585e666a757-service-ca-bundle" pod "647f31b7-3d07-41ac-925f-6585e666a757" (UID: "647f31b7-3d07-41ac-925f-6585e666a757"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:56:58.447603 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:58.447550 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/647f31b7-3d07-41ac-925f-6585e666a757-archives-path" pod "647f31b7-3d07-41ac-925f-6585e666a757" (UID: "647f31b7-3d07-41ac-925f-6585e666a757"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:56:58.447664 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:58.447639 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/647f31b7-3d07-41ac-925f-6585e666a757-trusted-ca-bundle" pod "647f31b7-3d07-41ac-925f-6585e666a757" (UID: "647f31b7-3d07-41ac-925f-6585e666a757"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:56:58.449366 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:58.449344 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/647f31b7-3d07-41ac-925f-6585e666a757-kube-api-access-4gphc" pod "647f31b7-3d07-41ac-925f-6585e666a757" (UID: "647f31b7-3d07-41ac-925f-6585e666a757"). InnerVolumeSpecName "kube-api-access-4gphc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:56:58.548577 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:58.548518 5054 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/647f31b7-3d07-41ac-925f-6585e666a757-archives-path\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:56:58.548577 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:58.548541 5054 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gphc\" (UniqueName: \"kubernetes.io/projected/647f31b7-3d07-41ac-925f-6585e666a757-kube-api-access-4gphc\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:56:58.548577 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:58.548553 5054 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/647f31b7-3d07-41ac-925f-6585e666a757-trusted-ca-bundle\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:56:58.548577 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:58.548563 5054 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/647f31b7-3d07-41ac-925f-6585e666a757-service-ca-bundle\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:56:59.256532 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:59.256497 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-6x2g6-kxhlf" event={"ID":"647f31b7-3d07-41ac-925f-6585e666a757","Type":"ContainerDied","Data":"4fc5172851c9c589f883d227c290ca3fd0cabfc3a295c250042c27f2675f910f"} Oct 30 18:56:59.256532 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:59.256531 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-6x2g6-kxhlf" Oct 30 18:56:59.256747 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:56:59.256544 5054 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4fc5172851c9c589f883d227c290ca3fd0cabfc3a295c250042c27f2675f910f" Oct 30 18:57:01.126114 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:57:01.126075 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" podUID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused" Oct 30 18:57:08.597106 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:57:08.597065 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:57:11.126203 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:57:11.126168 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" podUID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused" Oct 30 18:57:21.126637 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:57:21.126592 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" podUID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused" Oct 30 18:57:31.127143 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:57:31.127116 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:57:38.522887 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:57:38.522789 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 18:57:38.527261 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:57:38.527244 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 18:57:38.596859 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:57:38.596822 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:58:04.063147 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.063113 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x"] Oct 30 18:58:04.063586 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.063454 5054 cpu_manager.go:414] "containerMap: removing container" podUID="647f31b7-3d07-41ac-925f-6585e666a757" containerName="insights-gathering" Oct 30 18:58:04.063586 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.063465 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="647f31b7-3d07-41ac-925f-6585e666a757" containerName="insights-gathering" Oct 30 18:58:04.063586 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.063538 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="647f31b7-3d07-41ac-925f-6585e666a757" containerName="insights-gathering" Oct 30 18:58:04.067077 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.067056 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:58:04.157166 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.157131 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1855bf02-a5c7-430c-8d88-b837795f7681-kserve-provision-location\") pod \"isvc-canary-stop-v2-predictor-95597dd77-rmc5x\" (UID: \"1855bf02-a5c7-430c-8d88-b837795f7681\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:58:04.157328 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.157180 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkklm\" (UniqueName: \"kubernetes.io/projected/1855bf02-a5c7-430c-8d88-b837795f7681-kube-api-access-xkklm\") pod \"isvc-canary-stop-v2-predictor-95597dd77-rmc5x\" (UID: \"1855bf02-a5c7-430c-8d88-b837795f7681\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:58:04.157328 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.157204 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1855bf02-a5c7-430c-8d88-b837795f7681-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-v2-predictor-95597dd77-rmc5x\" (UID: \"1855bf02-a5c7-430c-8d88-b837795f7681\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:58:04.157421 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.157344 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1855bf02-a5c7-430c-8d88-b837795f7681-proxy-tls\") pod \"isvc-canary-stop-v2-predictor-95597dd77-rmc5x\" (UID: \"1855bf02-a5c7-430c-8d88-b837795f7681\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:58:04.157421 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.157413 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1855bf02-a5c7-430c-8d88-b837795f7681-cabundle-cert\") pod \"isvc-canary-stop-v2-predictor-95597dd77-rmc5x\" (UID: \"1855bf02-a5c7-430c-8d88-b837795f7681\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:58:04.161455 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.161409 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"seaweedfs-s3-creds\"" Oct 30 18:58:04.166477 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.166463 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"isvc-canary-stop-v2-predictor-serving-cert\"" Oct 30 18:58:04.166549 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.166475 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"default-dockercfg-zc4kv\"" Oct 30 18:58:04.166549 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.166532 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"kube-root-ca.crt\"" Oct 30 18:58:04.166621 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.166583 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"odh-kserve-custom-ca-bundle\"" Oct 30 18:58:04.166705 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.166683 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"openshift-service-ca.crt\"" Oct 30 18:58:04.166705 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.166686 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"isvc-canary-stop-kube-rbac-proxy-sar-config\"" Oct 30 18:58:04.168138 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.168123 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x"] Oct 30 18:58:04.258601 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.258573 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1855bf02-a5c7-430c-8d88-b837795f7681-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-v2-predictor-95597dd77-rmc5x\" (UID: \"1855bf02-a5c7-430c-8d88-b837795f7681\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:58:04.258743 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.258636 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1855bf02-a5c7-430c-8d88-b837795f7681-proxy-tls\") pod \"isvc-canary-stop-v2-predictor-95597dd77-rmc5x\" (UID: \"1855bf02-a5c7-430c-8d88-b837795f7681\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:58:04.258743 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.258685 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1855bf02-a5c7-430c-8d88-b837795f7681-cabundle-cert\") pod \"isvc-canary-stop-v2-predictor-95597dd77-rmc5x\" (UID: \"1855bf02-a5c7-430c-8d88-b837795f7681\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:58:04.258743 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.258717 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1855bf02-a5c7-430c-8d88-b837795f7681-kserve-provision-location\") pod \"isvc-canary-stop-v2-predictor-95597dd77-rmc5x\" (UID: \"1855bf02-a5c7-430c-8d88-b837795f7681\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:58:04.258866 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.258755 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkklm\" (UniqueName: \"kubernetes.io/projected/1855bf02-a5c7-430c-8d88-b837795f7681-kube-api-access-xkklm\") pod \"isvc-canary-stop-v2-predictor-95597dd77-rmc5x\" (UID: \"1855bf02-a5c7-430c-8d88-b837795f7681\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:58:04.259174 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.259138 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1855bf02-a5c7-430c-8d88-b837795f7681-kserve-provision-location\") pod \"isvc-canary-stop-v2-predictor-95597dd77-rmc5x\" (UID: \"1855bf02-a5c7-430c-8d88-b837795f7681\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:58:04.259358 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.259339 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1855bf02-a5c7-430c-8d88-b837795f7681-cabundle-cert\") pod \"isvc-canary-stop-v2-predictor-95597dd77-rmc5x\" (UID: \"1855bf02-a5c7-430c-8d88-b837795f7681\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:58:04.259438 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.259384 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1855bf02-a5c7-430c-8d88-b837795f7681-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-v2-predictor-95597dd77-rmc5x\" (UID: \"1855bf02-a5c7-430c-8d88-b837795f7681\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:58:04.261083 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.261065 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1855bf02-a5c7-430c-8d88-b837795f7681-proxy-tls\") pod \"isvc-canary-stop-v2-predictor-95597dd77-rmc5x\" (UID: \"1855bf02-a5c7-430c-8d88-b837795f7681\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:58:04.341387 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.341332 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkklm\" (UniqueName: \"kubernetes.io/projected/1855bf02-a5c7-430c-8d88-b837795f7681-kube-api-access-xkklm\") pod \"isvc-canary-stop-v2-predictor-95597dd77-rmc5x\" (UID: \"1855bf02-a5c7-430c-8d88-b837795f7681\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:58:04.383410 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.383381 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:58:04.651951 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.651925 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk"] Oct 30 18:58:04.655640 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.655621 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:58:04.708060 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.708041 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"isvc-canary-stop-predictor-serving-cert\"" Oct 30 18:58:04.712893 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.712867 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk"] Oct 30 18:58:04.762283 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.762256 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-proxy-tls\") pod \"isvc-canary-stop-predictor-594688d7b8-ggghk\" (UID: \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:58:04.762403 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.762290 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-594688d7b8-ggghk\" (UID: \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:58:04.762403 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.762382 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-kserve-provision-location\") pod \"isvc-canary-stop-predictor-594688d7b8-ggghk\" (UID: \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:58:04.762485 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.762433 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-cabundle-cert\") pod \"isvc-canary-stop-predictor-594688d7b8-ggghk\" (UID: \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:58:04.762485 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.762464 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gkmh\" (UniqueName: \"kubernetes.io/projected/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-kube-api-access-9gkmh\") pod \"isvc-canary-stop-predictor-594688d7b8-ggghk\" (UID: \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:58:04.777105 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.777082 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x"] Oct 30 18:58:04.783075 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:58:04.783052 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1855bf02_a5c7_430c_8d88_b837795f7681.slice/crio-f13cba5e79a03c9608df2b52236a3c9e03903dee03f93497ecec526e4bec138d WatchSource:0}: Error finding container f13cba5e79a03c9608df2b52236a3c9e03903dee03f93497ecec526e4bec138d: Status 404 returned error can't find the container with id f13cba5e79a03c9608df2b52236a3c9e03903dee03f93497ecec526e4bec138d Oct 30 18:58:04.863089 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.863056 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-kserve-provision-location\") pod \"isvc-canary-stop-predictor-594688d7b8-ggghk\" (UID: \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:58:04.863193 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.863147 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-cabundle-cert\") pod \"isvc-canary-stop-predictor-594688d7b8-ggghk\" (UID: \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:58:04.863257 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.863204 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gkmh\" (UniqueName: \"kubernetes.io/projected/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-kube-api-access-9gkmh\") pod \"isvc-canary-stop-predictor-594688d7b8-ggghk\" (UID: \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:58:04.863257 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.863244 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-proxy-tls\") pod \"isvc-canary-stop-predictor-594688d7b8-ggghk\" (UID: \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:58:04.863369 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.863279 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-594688d7b8-ggghk\" (UID: \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:58:04.863443 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.863415 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-kserve-provision-location\") pod \"isvc-canary-stop-predictor-594688d7b8-ggghk\" (UID: \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:58:04.863789 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.863769 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-cabundle-cert\") pod \"isvc-canary-stop-predictor-594688d7b8-ggghk\" (UID: \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:58:04.863887 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.863869 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-kube-rbac-proxy-sar-config\") pod \"isvc-canary-stop-predictor-594688d7b8-ggghk\" (UID: \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:58:04.865389 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.865369 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-proxy-tls\") pod \"isvc-canary-stop-predictor-594688d7b8-ggghk\" (UID: \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:58:04.924103 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.924050 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gkmh\" (UniqueName: \"kubernetes.io/projected/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-kube-api-access-9gkmh\") pod \"isvc-canary-stop-predictor-594688d7b8-ggghk\" (UID: \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\") " pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:58:04.971850 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:04.971824 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:58:05.350148 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:05.350074 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk"] Oct 30 18:58:05.356116 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:58:05.356084 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c7d6bb2_7e7f_425c_95e9_838c2d0ae678.slice/crio-4188a5b13065af78697960332bd4dd6190fb094077e85c7c8a80efe147dcbe84 WatchSource:0}: Error finding container 4188a5b13065af78697960332bd4dd6190fb094077e85c7c8a80efe147dcbe84: Status 404 returned error can't find the container with id 4188a5b13065af78697960332bd4dd6190fb094077e85c7c8a80efe147dcbe84 Oct 30 18:58:05.524854 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:05.524813 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" event={"ID":"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678","Type":"ContainerStarted","Data":"797ddd6f7f898b0f422bc4f7166efcc0eb42e6bab105a9928541aa2aad1dae9e"} Oct 30 18:58:05.525036 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:05.524884 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" event={"ID":"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678","Type":"ContainerStarted","Data":"4188a5b13065af78697960332bd4dd6190fb094077e85c7c8a80efe147dcbe84"} Oct 30 18:58:05.526511 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:05.526486 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" event={"ID":"1855bf02-a5c7-430c-8d88-b837795f7681","Type":"ContainerStarted","Data":"b2a5fda218da8a0df4e7ace52447836a247e2b91ad9d0c7d2f1ab75da7d165d0"} Oct 30 18:58:05.526640 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:05.526532 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" event={"ID":"1855bf02-a5c7-430c-8d88-b837795f7681","Type":"ContainerStarted","Data":"f13cba5e79a03c9608df2b52236a3c9e03903dee03f93497ecec526e4bec138d"} Oct 30 18:58:08.597387 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:58:08.597347 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:58:08.620635 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:08.620612 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq"] Oct 30 18:58:08.620853 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:08.620823 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" podUID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerName="kserve-container" containerID="cri-o://b91b0ecbb34a44f38c5c4f3f187c17bf59009740ebe3c4475147aafe30336850" gracePeriod=30 Oct 30 18:58:08.620952 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:08.620867 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" podUID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerName="kube-rbac-proxy" containerID="cri-o://3fb53fc7d7fbf2373cca41b8ae84302dd3024ff86647af75180c7fff9831833a" gracePeriod=30 Oct 30 18:58:09.544057 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:09.543956 5054 generic.go:356] "Generic (PLEG): container finished" podID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerID="797ddd6f7f898b0f422bc4f7166efcc0eb42e6bab105a9928541aa2aad1dae9e" exitCode=0 Oct 30 18:58:09.544057 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:09.544029 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" event={"ID":"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678","Type":"ContainerDied","Data":"797ddd6f7f898b0f422bc4f7166efcc0eb42e6bab105a9928541aa2aad1dae9e"} Oct 30 18:58:09.545835 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:09.545754 5054 generic.go:356] "Generic (PLEG): container finished" podID="1855bf02-a5c7-430c-8d88-b837795f7681" containerID="b2a5fda218da8a0df4e7ace52447836a247e2b91ad9d0c7d2f1ab75da7d165d0" exitCode=0 Oct 30 18:58:09.545835 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:09.545779 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" event={"ID":"1855bf02-a5c7-430c-8d88-b837795f7681","Type":"ContainerDied","Data":"b2a5fda218da8a0df4e7ace52447836a247e2b91ad9d0c7d2f1ab75da7d165d0"} Oct 30 18:58:09.548600 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:09.548525 5054 generic.go:356] "Generic (PLEG): container finished" podID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerID="3fb53fc7d7fbf2373cca41b8ae84302dd3024ff86647af75180c7fff9831833a" exitCode=0 Oct 30 18:58:09.548600 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:09.548571 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" event={"ID":"bc6b5829-a6bc-4764-8051-63ef89fa9287","Type":"ContainerDied","Data":"3fb53fc7d7fbf2373cca41b8ae84302dd3024ff86647af75180c7fff9831833a"} Oct 30 18:58:10.556197 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:10.556164 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" event={"ID":"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678","Type":"ContainerStarted","Data":"33d9a1fd046bf629d21df06cdaedb59765618902442eb6be485e72aaae740070"} Oct 30 18:58:10.556627 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:10.556238 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" event={"ID":"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678","Type":"ContainerStarted","Data":"5cf0f857c0403de25757555a8a67dfe40da99b16ddeda7982b5b310bedda1c1e"} Oct 30 18:58:10.556627 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:10.556442 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:58:10.561259 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:10.561234 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" event={"ID":"1855bf02-a5c7-430c-8d88-b837795f7681","Type":"ContainerStarted","Data":"1a9b2b0c843c6b0a65dc9e443e73b423e4fe99681ed019a7bdc869f28211c3cd"} Oct 30 18:58:10.561366 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:10.561284 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" event={"ID":"1855bf02-a5c7-430c-8d88-b837795f7681","Type":"ContainerStarted","Data":"6f759446be652d8ee47379b9f968c388c2c249f7f1189651404256d100a99dad"} Oct 30 18:58:10.561482 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:10.561465 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:58:10.561525 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:10.561515 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:58:10.562681 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:10.562658 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" podUID="1855bf02-a5c7-430c-8d88-b837795f7681" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused" Oct 30 18:58:10.674408 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:10.674359 5054 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" podStartSLOduration=2.96503172 podStartE2EDuration="6.674345801s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.709314081s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:10.67409672 +0000 UTC m=+1232.402475431" watchObservedRunningTime="2026-10-30 18:58:10.674345801 +0000 UTC m=+1232.402724517" Oct 30 18:58:10.793691 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:10.793636 5054 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" podStartSLOduration=2.904691113 podStartE2EDuration="6.793622474s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.888931361s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:10.791952305 +0000 UTC m=+1232.520331020" watchObservedRunningTime="2026-10-30 18:58:10.793622474 +0000 UTC m=+1232.522001189" Oct 30 18:58:11.122656 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:11.122613 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" podUID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.38:8643/healthz\": dial tcp 10.132.0.38:8643: connect: connection refused" Oct 30 18:58:11.127040 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:11.126992 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" podUID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.38:8080: connect: connection refused" Oct 30 18:58:11.571675 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:11.571577 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" podUID="1855bf02-a5c7-430c-8d88-b837795f7681" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused" Oct 30 18:58:11.572116 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:11.571791 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:58:11.572781 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:11.572758 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" podUID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused" Oct 30 18:58:12.573446 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.573422 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:58:12.576950 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.576926 5054 generic.go:356] "Generic (PLEG): container finished" podID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerID="b91b0ecbb34a44f38c5c4f3f187c17bf59009740ebe3c4475147aafe30336850" exitCode=0 Oct 30 18:58:12.577079 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.576976 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" Oct 30 18:58:12.577079 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.577017 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" event={"ID":"bc6b5829-a6bc-4764-8051-63ef89fa9287","Type":"ContainerDied","Data":"b91b0ecbb34a44f38c5c4f3f187c17bf59009740ebe3c4475147aafe30336850"} Oct 30 18:58:12.577079 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.577073 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq" event={"ID":"bc6b5829-a6bc-4764-8051-63ef89fa9287","Type":"ContainerDied","Data":"3adf2e769e6ccaed4179f30504c121ad6af58123111154c12a995c154ecf328f"} Oct 30 18:58:12.577178 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.577107 5054 scope.go:122] "RemoveContainer" containerID="3fb53fc7d7fbf2373cca41b8ae84302dd3024ff86647af75180c7fff9831833a" Oct 30 18:58:12.577566 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.577528 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" podUID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused" Oct 30 18:58:12.585052 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.584996 5054 scope.go:122] "RemoveContainer" containerID="b91b0ecbb34a44f38c5c4f3f187c17bf59009740ebe3c4475147aafe30336850" Oct 30 18:58:12.592293 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.592277 5054 scope.go:122] "RemoveContainer" containerID="9234af7a89559eda38b98265acaeb6103a649edbc05616f49754565538a038f7" Oct 30 18:58:12.599246 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.599228 5054 scope.go:122] "RemoveContainer" containerID="3fb53fc7d7fbf2373cca41b8ae84302dd3024ff86647af75180c7fff9831833a" Oct 30 18:58:12.599518 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:58:12.599500 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fb53fc7d7fbf2373cca41b8ae84302dd3024ff86647af75180c7fff9831833a\": container with ID starting with 3fb53fc7d7fbf2373cca41b8ae84302dd3024ff86647af75180c7fff9831833a not found: ID does not exist" containerID="3fb53fc7d7fbf2373cca41b8ae84302dd3024ff86647af75180c7fff9831833a" Oct 30 18:58:12.599565 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.599527 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fb53fc7d7fbf2373cca41b8ae84302dd3024ff86647af75180c7fff9831833a"} err="failed to get container status \"3fb53fc7d7fbf2373cca41b8ae84302dd3024ff86647af75180c7fff9831833a\": rpc error: code = NotFound desc = could not find container \"3fb53fc7d7fbf2373cca41b8ae84302dd3024ff86647af75180c7fff9831833a\": container with ID starting with 3fb53fc7d7fbf2373cca41b8ae84302dd3024ff86647af75180c7fff9831833a not found: ID does not exist" Oct 30 18:58:12.599565 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.599542 5054 scope.go:122] "RemoveContainer" containerID="b91b0ecbb34a44f38c5c4f3f187c17bf59009740ebe3c4475147aafe30336850" Oct 30 18:58:12.599762 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:58:12.599745 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b91b0ecbb34a44f38c5c4f3f187c17bf59009740ebe3c4475147aafe30336850\": container with ID starting with b91b0ecbb34a44f38c5c4f3f187c17bf59009740ebe3c4475147aafe30336850 not found: ID does not exist" containerID="b91b0ecbb34a44f38c5c4f3f187c17bf59009740ebe3c4475147aafe30336850" Oct 30 18:58:12.599829 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.599773 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b91b0ecbb34a44f38c5c4f3f187c17bf59009740ebe3c4475147aafe30336850"} err="failed to get container status \"b91b0ecbb34a44f38c5c4f3f187c17bf59009740ebe3c4475147aafe30336850\": rpc error: code = NotFound desc = could not find container \"b91b0ecbb34a44f38c5c4f3f187c17bf59009740ebe3c4475147aafe30336850\": container with ID starting with b91b0ecbb34a44f38c5c4f3f187c17bf59009740ebe3c4475147aafe30336850 not found: ID does not exist" Oct 30 18:58:12.599829 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.599795 5054 scope.go:122] "RemoveContainer" containerID="9234af7a89559eda38b98265acaeb6103a649edbc05616f49754565538a038f7" Oct 30 18:58:12.600097 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:58:12.599991 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9234af7a89559eda38b98265acaeb6103a649edbc05616f49754565538a038f7\": container with ID starting with 9234af7a89559eda38b98265acaeb6103a649edbc05616f49754565538a038f7 not found: ID does not exist" containerID="9234af7a89559eda38b98265acaeb6103a649edbc05616f49754565538a038f7" Oct 30 18:58:12.600097 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.600022 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9234af7a89559eda38b98265acaeb6103a649edbc05616f49754565538a038f7"} err="failed to get container status \"9234af7a89559eda38b98265acaeb6103a649edbc05616f49754565538a038f7\": rpc error: code = NotFound desc = could not find container \"9234af7a89559eda38b98265acaeb6103a649edbc05616f49754565538a038f7\": container with ID starting with 9234af7a89559eda38b98265acaeb6103a649edbc05616f49754565538a038f7 not found: ID does not exist" Oct 30 18:58:12.730406 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.730367 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc6b5829-a6bc-4764-8051-63ef89fa9287-kube-api-access-6gqwd\" (UniqueName: \"kubernetes.io/projected/bc6b5829-a6bc-4764-8051-63ef89fa9287-kube-api-access-6gqwd\") pod \"bc6b5829-a6bc-4764-8051-63ef89fa9287\" (UID: \"bc6b5829-a6bc-4764-8051-63ef89fa9287\") " Oct 30 18:58:12.730406 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.730414 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc6b5829-a6bc-4764-8051-63ef89fa9287-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bc6b5829-a6bc-4764-8051-63ef89fa9287-kube-rbac-proxy-sar-config\") pod \"bc6b5829-a6bc-4764-8051-63ef89fa9287\" (UID: \"bc6b5829-a6bc-4764-8051-63ef89fa9287\") " Oct 30 18:58:12.730591 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.730449 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bc6b5829-a6bc-4764-8051-63ef89fa9287-proxy-tls\" (UniqueName: \"kubernetes.io/secret/bc6b5829-a6bc-4764-8051-63ef89fa9287-proxy-tls\") pod \"bc6b5829-a6bc-4764-8051-63ef89fa9287\" (UID: \"bc6b5829-a6bc-4764-8051-63ef89fa9287\") " Oct 30 18:58:12.730591 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.730481 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc6b5829-a6bc-4764-8051-63ef89fa9287-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bc6b5829-a6bc-4764-8051-63ef89fa9287-cabundle-cert\") pod \"bc6b5829-a6bc-4764-8051-63ef89fa9287\" (UID: \"bc6b5829-a6bc-4764-8051-63ef89fa9287\") " Oct 30 18:58:12.730591 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.730502 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc6b5829-a6bc-4764-8051-63ef89fa9287-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bc6b5829-a6bc-4764-8051-63ef89fa9287-kserve-provision-location\") pod \"bc6b5829-a6bc-4764-8051-63ef89fa9287\" (UID: \"bc6b5829-a6bc-4764-8051-63ef89fa9287\") " Oct 30 18:58:12.730901 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.730872 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc6b5829-a6bc-4764-8051-63ef89fa9287-cabundle-cert" pod "bc6b5829-a6bc-4764-8051-63ef89fa9287" (UID: "bc6b5829-a6bc-4764-8051-63ef89fa9287"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:58:12.731049 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.730869 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc6b5829-a6bc-4764-8051-63ef89fa9287-kube-rbac-proxy-sar-config" pod "bc6b5829-a6bc-4764-8051-63ef89fa9287" (UID: "bc6b5829-a6bc-4764-8051-63ef89fa9287"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:58:12.731049 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.730914 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc6b5829-a6bc-4764-8051-63ef89fa9287-kserve-provision-location" pod "bc6b5829-a6bc-4764-8051-63ef89fa9287" (UID: "bc6b5829-a6bc-4764-8051-63ef89fa9287"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:58:12.732569 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.732544 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bc6b5829-a6bc-4764-8051-63ef89fa9287-proxy-tls" pod "bc6b5829-a6bc-4764-8051-63ef89fa9287" (UID: "bc6b5829-a6bc-4764-8051-63ef89fa9287"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:58:12.732650 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.732605 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc6b5829-a6bc-4764-8051-63ef89fa9287-kube-api-access-6gqwd" pod "bc6b5829-a6bc-4764-8051-63ef89fa9287" (UID: "bc6b5829-a6bc-4764-8051-63ef89fa9287"). InnerVolumeSpecName "kube-api-access-6gqwd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:58:12.831449 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.831416 5054 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bc6b5829-a6bc-4764-8051-63ef89fa9287-proxy-tls\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:58:12.831449 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.831444 5054 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bc6b5829-a6bc-4764-8051-63ef89fa9287-cabundle-cert\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:58:12.831643 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.831458 5054 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bc6b5829-a6bc-4764-8051-63ef89fa9287-kserve-provision-location\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:58:12.831643 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.831473 5054 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gqwd\" (UniqueName: \"kubernetes.io/projected/bc6b5829-a6bc-4764-8051-63ef89fa9287-kube-api-access-6gqwd\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:58:12.831643 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.831485 5054 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bc6b5829-a6bc-4764-8051-63ef89fa9287-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:58:12.996489 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:12.996459 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq"] Oct 30 18:58:13.002862 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:13.002834 5054 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw0-1fd37956ac/isvc-xgboost-graph-raw-hpa-6d75d-predictor-846c6f589b-xnjrq"] Oct 30 18:58:14.421479 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:14.421449 5054 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bc6b5829-a6bc-4764-8051-63ef89fa9287" path="/var/lib/kubelet/pods/bc6b5829-a6bc-4764-8051-63ef89fa9287/volumes" Oct 30 18:58:16.575760 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:16.575734 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:58:16.576208 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:16.576184 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" podUID="1855bf02-a5c7-430c-8d88-b837795f7681" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused" Oct 30 18:58:17.582020 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:17.581973 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:58:17.582570 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:17.582532 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" podUID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused" Oct 30 18:58:26.576679 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:26.576640 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" podUID="1855bf02-a5c7-430c-8d88-b837795f7681" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused" Oct 30 18:58:27.582730 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:27.582696 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" podUID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused" Oct 30 18:58:36.576887 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:36.576845 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" podUID="1855bf02-a5c7-430c-8d88-b837795f7681" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused" Oct 30 18:58:37.582496 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:37.582459 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" podUID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused" Oct 30 18:58:38.596822 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:58:38.596786 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:58:46.577128 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:46.577091 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" podUID="1855bf02-a5c7-430c-8d88-b837795f7681" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused" Oct 30 18:58:47.582671 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:47.582631 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" podUID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused" Oct 30 18:58:56.576383 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:56.576342 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" podUID="1855bf02-a5c7-430c-8d88-b837795f7681" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused" Oct 30 18:58:57.582849 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:58:57.582808 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" podUID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused" Oct 30 18:59:06.576792 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:06.576754 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" podUID="1855bf02-a5c7-430c-8d88-b837795f7681" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused" Oct 30 18:59:07.583268 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:07.583227 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" podUID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused" Oct 30 18:59:08.599256 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:59:08.599223 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:59:16.576953 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:16.576917 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" podUID="1855bf02-a5c7-430c-8d88-b837795f7681" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused" Oct 30 18:59:17.582628 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:17.582590 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" podUID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused" Oct 30 18:59:26.576962 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:26.576892 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:59:27.583745 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:27.583710 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:59:33.214928 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:33.214896 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x"] Oct 30 18:59:33.215363 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:33.215138 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" podUID="1855bf02-a5c7-430c-8d88-b837795f7681" containerName="kserve-container" containerID="cri-o://6f759446be652d8ee47379b9f968c388c2c249f7f1189651404256d100a99dad" gracePeriod=30 Oct 30 18:59:33.215363 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:33.215184 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" podUID="1855bf02-a5c7-430c-8d88-b837795f7681" containerName="kube-rbac-proxy" containerID="cri-o://1a9b2b0c843c6b0a65dc9e443e73b423e4fe99681ed019a7bdc869f28211c3cd" gracePeriod=30 Oct 30 18:59:33.734061 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:33.734031 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk"] Oct 30 18:59:33.734276 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:33.734252 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" podUID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerName="kserve-container" containerID="cri-o://5cf0f857c0403de25757555a8a67dfe40da99b16ddeda7982b5b310bedda1c1e" gracePeriod=30 Oct 30 18:59:33.734351 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:33.734302 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" podUID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerName="kube-rbac-proxy" containerID="cri-o://33d9a1fd046bf629d21df06cdaedb59765618902442eb6be485e72aaae740070" gracePeriod=30 Oct 30 18:59:33.925158 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:33.925126 5054 generic.go:356] "Generic (PLEG): container finished" podID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerID="33d9a1fd046bf629d21df06cdaedb59765618902442eb6be485e72aaae740070" exitCode=0 Oct 30 18:59:33.925334 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:33.925197 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" event={"ID":"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678","Type":"ContainerDied","Data":"33d9a1fd046bf629d21df06cdaedb59765618902442eb6be485e72aaae740070"} Oct 30 18:59:33.927373 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:33.927346 5054 generic.go:356] "Generic (PLEG): container finished" podID="1855bf02-a5c7-430c-8d88-b837795f7681" containerID="1a9b2b0c843c6b0a65dc9e443e73b423e4fe99681ed019a7bdc869f28211c3cd" exitCode=0 Oct 30 18:59:33.927487 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:33.927401 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" event={"ID":"1855bf02-a5c7-430c-8d88-b837795f7681","Type":"ContainerDied","Data":"1a9b2b0c843c6b0a65dc9e443e73b423e4fe99681ed019a7bdc869f28211c3cd"} Oct 30 18:59:36.571682 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:36.571641 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" podUID="1855bf02-a5c7-430c-8d88-b837795f7681" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.39:8643/healthz\": dial tcp 10.132.0.39:8643: connect: connection refused" Oct 30 18:59:36.576925 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:36.576893 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" podUID="1855bf02-a5c7-430c-8d88-b837795f7681" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.39:8080: connect: connection refused" Oct 30 18:59:37.577736 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:37.577702 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" podUID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.40:8643/healthz\": dial tcp 10.132.0.40:8643: connect: connection refused" Oct 30 18:59:37.583043 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:37.583016 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" podUID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.40:8080: connect: connection refused" Oct 30 18:59:37.947443 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:37.947398 5054 generic.go:356] "Generic (PLEG): container finished" podID="1855bf02-a5c7-430c-8d88-b837795f7681" containerID="6f759446be652d8ee47379b9f968c388c2c249f7f1189651404256d100a99dad" exitCode=0 Oct 30 18:59:37.947599 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:37.947442 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" event={"ID":"1855bf02-a5c7-430c-8d88-b837795f7681","Type":"ContainerDied","Data":"6f759446be652d8ee47379b9f968c388c2c249f7f1189651404256d100a99dad"} Oct 30 18:59:38.062858 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.062834 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:59:38.136594 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.136568 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1855bf02-a5c7-430c-8d88-b837795f7681-proxy-tls\" (UniqueName: \"kubernetes.io/secret/1855bf02-a5c7-430c-8d88-b837795f7681-proxy-tls\") pod \"1855bf02-a5c7-430c-8d88-b837795f7681\" (UID: \"1855bf02-a5c7-430c-8d88-b837795f7681\") " Oct 30 18:59:38.136720 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.136608 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1855bf02-a5c7-430c-8d88-b837795f7681-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1855bf02-a5c7-430c-8d88-b837795f7681-kserve-provision-location\") pod \"1855bf02-a5c7-430c-8d88-b837795f7681\" (UID: \"1855bf02-a5c7-430c-8d88-b837795f7681\") " Oct 30 18:59:38.136720 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.136652 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1855bf02-a5c7-430c-8d88-b837795f7681-kube-api-access-xkklm\" (UniqueName: \"kubernetes.io/projected/1855bf02-a5c7-430c-8d88-b837795f7681-kube-api-access-xkklm\") pod \"1855bf02-a5c7-430c-8d88-b837795f7681\" (UID: \"1855bf02-a5c7-430c-8d88-b837795f7681\") " Oct 30 18:59:38.136720 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.136678 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1855bf02-a5c7-430c-8d88-b837795f7681-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1855bf02-a5c7-430c-8d88-b837795f7681-cabundle-cert\") pod \"1855bf02-a5c7-430c-8d88-b837795f7681\" (UID: \"1855bf02-a5c7-430c-8d88-b837795f7681\") " Oct 30 18:59:38.136720 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.136717 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1855bf02-a5c7-430c-8d88-b837795f7681-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1855bf02-a5c7-430c-8d88-b837795f7681-kube-rbac-proxy-sar-config\") pod \"1855bf02-a5c7-430c-8d88-b837795f7681\" (UID: \"1855bf02-a5c7-430c-8d88-b837795f7681\") " Oct 30 18:59:38.137077 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.137046 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1855bf02-a5c7-430c-8d88-b837795f7681-kserve-provision-location" pod "1855bf02-a5c7-430c-8d88-b837795f7681" (UID: "1855bf02-a5c7-430c-8d88-b837795f7681"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:59:38.137174 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.137087 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1855bf02-a5c7-430c-8d88-b837795f7681-cabundle-cert" pod "1855bf02-a5c7-430c-8d88-b837795f7681" (UID: "1855bf02-a5c7-430c-8d88-b837795f7681"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:59:38.137174 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.137120 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1855bf02-a5c7-430c-8d88-b837795f7681-kube-rbac-proxy-sar-config" pod "1855bf02-a5c7-430c-8d88-b837795f7681" (UID: "1855bf02-a5c7-430c-8d88-b837795f7681"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:59:38.138667 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.138640 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1855bf02-a5c7-430c-8d88-b837795f7681-kube-api-access-xkklm" pod "1855bf02-a5c7-430c-8d88-b837795f7681" (UID: "1855bf02-a5c7-430c-8d88-b837795f7681"). InnerVolumeSpecName "kube-api-access-xkklm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:59:38.138744 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.138714 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1855bf02-a5c7-430c-8d88-b837795f7681-proxy-tls" pod "1855bf02-a5c7-430c-8d88-b837795f7681" (UID: "1855bf02-a5c7-430c-8d88-b837795f7681"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:59:38.237879 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.237852 5054 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1855bf02-a5c7-430c-8d88-b837795f7681-proxy-tls\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:59:38.237879 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.237876 5054 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1855bf02-a5c7-430c-8d88-b837795f7681-kserve-provision-location\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:59:38.237879 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.237889 5054 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkklm\" (UniqueName: \"kubernetes.io/projected/1855bf02-a5c7-430c-8d88-b837795f7681-kube-api-access-xkklm\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:59:38.238172 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.237903 5054 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1855bf02-a5c7-430c-8d88-b837795f7681-cabundle-cert\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:59:38.238172 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.237916 5054 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1855bf02-a5c7-430c-8d88-b837795f7681-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:59:38.565958 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.565936 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:59:38.597980 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:59:38.597945 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 18:59:38.640242 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.640219 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-cabundle-cert\") pod \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\" (UID: \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\") " Oct 30 18:59:38.640384 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.640257 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-kserve-provision-location\") pod \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\" (UID: \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\") " Oct 30 18:59:38.640384 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.640319 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-kube-rbac-proxy-sar-config\") pod \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\" (UID: \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\") " Oct 30 18:59:38.640384 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.640360 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-proxy-tls\" (UniqueName: \"kubernetes.io/secret/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-proxy-tls\") pod \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\" (UID: \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\") " Oct 30 18:59:38.640556 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.640397 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-kube-api-access-9gkmh\" (UniqueName: \"kubernetes.io/projected/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-kube-api-access-9gkmh\") pod \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\" (UID: \"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678\") " Oct 30 18:59:38.640669 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.640646 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-cabundle-cert" pod "5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" (UID: "5c7d6bb2-7e7f-425c-95e9-838c2d0ae678"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:59:38.640721 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.640652 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-kserve-provision-location" pod "5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" (UID: "5c7d6bb2-7e7f-425c-95e9-838c2d0ae678"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:59:38.640756 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.640720 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-kube-rbac-proxy-sar-config" pod "5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" (UID: "5c7d6bb2-7e7f-425c-95e9-838c2d0ae678"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:59:38.642372 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.642350 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-proxy-tls" pod "5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" (UID: "5c7d6bb2-7e7f-425c-95e9-838c2d0ae678"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:59:38.642437 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.642414 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-kube-api-access-9gkmh" pod "5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" (UID: "5c7d6bb2-7e7f-425c-95e9-838c2d0ae678"). InnerVolumeSpecName "kube-api-access-9gkmh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:59:38.741887 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.741807 5054 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gkmh\" (UniqueName: \"kubernetes.io/projected/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-kube-api-access-9gkmh\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:59:38.741887 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.741834 5054 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-cabundle-cert\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:59:38.741887 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.741845 5054 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-kserve-provision-location\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:59:38.741887 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.741855 5054 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:59:38.741887 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.741864 5054 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678-proxy-tls\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 18:59:38.953395 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.953364 5054 generic.go:356] "Generic (PLEG): container finished" podID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerID="5cf0f857c0403de25757555a8a67dfe40da99b16ddeda7982b5b310bedda1c1e" exitCode=0 Oct 30 18:59:38.953543 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.953424 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" Oct 30 18:59:38.953543 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.953447 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" event={"ID":"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678","Type":"ContainerDied","Data":"5cf0f857c0403de25757555a8a67dfe40da99b16ddeda7982b5b310bedda1c1e"} Oct 30 18:59:38.953543 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.953520 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk" event={"ID":"5c7d6bb2-7e7f-425c-95e9-838c2d0ae678","Type":"ContainerDied","Data":"4188a5b13065af78697960332bd4dd6190fb094077e85c7c8a80efe147dcbe84"} Oct 30 18:59:38.953663 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.953557 5054 scope.go:122] "RemoveContainer" containerID="33d9a1fd046bf629d21df06cdaedb59765618902442eb6be485e72aaae740070" Oct 30 18:59:38.955448 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.955423 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" event={"ID":"1855bf02-a5c7-430c-8d88-b837795f7681","Type":"ContainerDied","Data":"f13cba5e79a03c9608df2b52236a3c9e03903dee03f93497ecec526e4bec138d"} Oct 30 18:59:38.955575 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.955455 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x" Oct 30 18:59:38.962515 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.962495 5054 scope.go:122] "RemoveContainer" containerID="5cf0f857c0403de25757555a8a67dfe40da99b16ddeda7982b5b310bedda1c1e" Oct 30 18:59:38.969483 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.969463 5054 scope.go:122] "RemoveContainer" containerID="797ddd6f7f898b0f422bc4f7166efcc0eb42e6bab105a9928541aa2aad1dae9e" Oct 30 18:59:38.978840 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.978822 5054 scope.go:122] "RemoveContainer" containerID="33d9a1fd046bf629d21df06cdaedb59765618902442eb6be485e72aaae740070" Oct 30 18:59:38.979141 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:59:38.979114 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33d9a1fd046bf629d21df06cdaedb59765618902442eb6be485e72aaae740070\": container with ID starting with 33d9a1fd046bf629d21df06cdaedb59765618902442eb6be485e72aaae740070 not found: ID does not exist" containerID="33d9a1fd046bf629d21df06cdaedb59765618902442eb6be485e72aaae740070" Oct 30 18:59:38.979230 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.979150 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33d9a1fd046bf629d21df06cdaedb59765618902442eb6be485e72aaae740070"} err="failed to get container status \"33d9a1fd046bf629d21df06cdaedb59765618902442eb6be485e72aaae740070\": rpc error: code = NotFound desc = could not find container \"33d9a1fd046bf629d21df06cdaedb59765618902442eb6be485e72aaae740070\": container with ID starting with 33d9a1fd046bf629d21df06cdaedb59765618902442eb6be485e72aaae740070 not found: ID does not exist" Oct 30 18:59:38.979230 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.979175 5054 scope.go:122] "RemoveContainer" containerID="5cf0f857c0403de25757555a8a67dfe40da99b16ddeda7982b5b310bedda1c1e" Oct 30 18:59:38.979480 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:59:38.979462 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cf0f857c0403de25757555a8a67dfe40da99b16ddeda7982b5b310bedda1c1e\": container with ID starting with 5cf0f857c0403de25757555a8a67dfe40da99b16ddeda7982b5b310bedda1c1e not found: ID does not exist" containerID="5cf0f857c0403de25757555a8a67dfe40da99b16ddeda7982b5b310bedda1c1e" Oct 30 18:59:38.979525 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.979489 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cf0f857c0403de25757555a8a67dfe40da99b16ddeda7982b5b310bedda1c1e"} err="failed to get container status \"5cf0f857c0403de25757555a8a67dfe40da99b16ddeda7982b5b310bedda1c1e\": rpc error: code = NotFound desc = could not find container \"5cf0f857c0403de25757555a8a67dfe40da99b16ddeda7982b5b310bedda1c1e\": container with ID starting with 5cf0f857c0403de25757555a8a67dfe40da99b16ddeda7982b5b310bedda1c1e not found: ID does not exist" Oct 30 18:59:38.979525 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.979508 5054 scope.go:122] "RemoveContainer" containerID="797ddd6f7f898b0f422bc4f7166efcc0eb42e6bab105a9928541aa2aad1dae9e" Oct 30 18:59:38.979741 ip-10-0-132-74 kubenswrapper[5054]: E1030 18:59:38.979722 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"797ddd6f7f898b0f422bc4f7166efcc0eb42e6bab105a9928541aa2aad1dae9e\": container with ID starting with 797ddd6f7f898b0f422bc4f7166efcc0eb42e6bab105a9928541aa2aad1dae9e not found: ID does not exist" containerID="797ddd6f7f898b0f422bc4f7166efcc0eb42e6bab105a9928541aa2aad1dae9e" Oct 30 18:59:38.979810 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.979749 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"797ddd6f7f898b0f422bc4f7166efcc0eb42e6bab105a9928541aa2aad1dae9e"} err="failed to get container status \"797ddd6f7f898b0f422bc4f7166efcc0eb42e6bab105a9928541aa2aad1dae9e\": rpc error: code = NotFound desc = could not find container \"797ddd6f7f898b0f422bc4f7166efcc0eb42e6bab105a9928541aa2aad1dae9e\": container with ID starting with 797ddd6f7f898b0f422bc4f7166efcc0eb42e6bab105a9928541aa2aad1dae9e not found: ID does not exist" Oct 30 18:59:38.979810 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.979772 5054 scope.go:122] "RemoveContainer" containerID="1a9b2b0c843c6b0a65dc9e443e73b423e4fe99681ed019a7bdc869f28211c3cd" Oct 30 18:59:38.986813 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.986798 5054 scope.go:122] "RemoveContainer" containerID="6f759446be652d8ee47379b9f968c388c2c249f7f1189651404256d100a99dad" Oct 30 18:59:38.993350 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:38.993335 5054 scope.go:122] "RemoveContainer" containerID="b2a5fda218da8a0df4e7ace52447836a247e2b91ad9d0c7d2f1ab75da7d165d0" Oct 30 18:59:39.117027 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:39.116987 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x"] Oct 30 18:59:39.164617 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:39.164588 5054 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-stop-v2-predictor-95597dd77-rmc5x"] Oct 30 18:59:39.232652 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:39.232621 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk"] Oct 30 18:59:39.238818 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:39.238792 5054 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-stop-predictor-594688d7b8-ggghk"] Oct 30 18:59:40.421046 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:40.420997 5054 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1855bf02-a5c7-430c-8d88-b837795f7681" path="/var/lib/kubelet/pods/1855bf02-a5c7-430c-8d88-b837795f7681/volumes" Oct 30 18:59:40.421509 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:40.421496 5054 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" path="/var/lib/kubelet/pods/5c7d6bb2-7e7f-425c-95e9-838c2d0ae678/volumes" Oct 30 18:59:43.223846 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.223806 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g"] Oct 30 18:59:43.224361 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224266 5054 cpu_manager.go:414] "containerMap: removing container" podUID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerName="kube-rbac-proxy" Oct 30 18:59:43.224361 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224285 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerName="kube-rbac-proxy" Oct 30 18:59:43.224361 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224296 5054 cpu_manager.go:414] "containerMap: removing container" podUID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerName="storage-initializer" Oct 30 18:59:43.224361 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224303 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerName="storage-initializer" Oct 30 18:59:43.224361 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224315 5054 cpu_manager.go:414] "containerMap: removing container" podUID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerName="kube-rbac-proxy" Oct 30 18:59:43.224361 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224323 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerName="kube-rbac-proxy" Oct 30 18:59:43.224361 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224345 5054 cpu_manager.go:414] "containerMap: removing container" podUID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerName="storage-initializer" Oct 30 18:59:43.224361 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224352 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerName="storage-initializer" Oct 30 18:59:43.224361 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224363 5054 cpu_manager.go:414] "containerMap: removing container" podUID="1855bf02-a5c7-430c-8d88-b837795f7681" containerName="kube-rbac-proxy" Oct 30 18:59:43.224822 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224370 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1855bf02-a5c7-430c-8d88-b837795f7681" containerName="kube-rbac-proxy" Oct 30 18:59:43.224822 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224385 5054 cpu_manager.go:414] "containerMap: removing container" podUID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerName="kserve-container" Oct 30 18:59:43.224822 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224392 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerName="kserve-container" Oct 30 18:59:43.224822 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224419 5054 cpu_manager.go:414] "containerMap: removing container" podUID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerName="kserve-container" Oct 30 18:59:43.224822 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224429 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerName="kserve-container" Oct 30 18:59:43.224822 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224443 5054 cpu_manager.go:414] "containerMap: removing container" podUID="1855bf02-a5c7-430c-8d88-b837795f7681" containerName="kserve-container" Oct 30 18:59:43.224822 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224452 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1855bf02-a5c7-430c-8d88-b837795f7681" containerName="kserve-container" Oct 30 18:59:43.224822 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224469 5054 cpu_manager.go:414] "containerMap: removing container" podUID="1855bf02-a5c7-430c-8d88-b837795f7681" containerName="storage-initializer" Oct 30 18:59:43.224822 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224476 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1855bf02-a5c7-430c-8d88-b837795f7681" containerName="storage-initializer" Oct 30 18:59:43.224822 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224581 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerName="kube-rbac-proxy" Oct 30 18:59:43.224822 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224594 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerName="kube-rbac-proxy" Oct 30 18:59:43.224822 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224607 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="1855bf02-a5c7-430c-8d88-b837795f7681" containerName="kube-rbac-proxy" Oct 30 18:59:43.224822 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224618 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc6b5829-a6bc-4764-8051-63ef89fa9287" containerName="kserve-container" Oct 30 18:59:43.224822 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224628 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c7d6bb2-7e7f-425c-95e9-838c2d0ae678" containerName="kserve-container" Oct 30 18:59:43.224822 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.224638 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="1855bf02-a5c7-430c-8d88-b837795f7681" containerName="kserve-container" Oct 30 18:59:43.229755 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.229733 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 18:59:43.280463 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.280430 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp248\" (UniqueName: \"kubernetes.io/projected/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-kube-api-access-tp248\") pod \"isvc-primary-29ca4c-predictor-858dbb99c6-7487g\" (UID: \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\") " pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 18:59:43.280463 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.280464 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-kube-rbac-proxy-sar-config\") pod \"isvc-primary-29ca4c-predictor-858dbb99c6-7487g\" (UID: \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\") " pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 18:59:43.280629 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.280605 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-kserve-provision-location\") pod \"isvc-primary-29ca4c-predictor-858dbb99c6-7487g\" (UID: \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\") " pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 18:59:43.280671 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.280657 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-cabundle-cert\") pod \"isvc-primary-29ca4c-predictor-858dbb99c6-7487g\" (UID: \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\") " pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 18:59:43.280708 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.280682 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-proxy-tls\") pod \"isvc-primary-29ca4c-predictor-858dbb99c6-7487g\" (UID: \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\") " pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 18:59:43.331167 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.331145 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"isvc-primary-29ca4c-predictor-serving-cert\"" Oct 30 18:59:43.331290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.331183 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"odh-kserve-custom-ca-bundle\"" Oct 30 18:59:43.331290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.331241 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"isvc-primary-29ca4c-kube-rbac-proxy-sar-config\"" Oct 30 18:59:43.331290 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.331271 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"seaweedfs-s3-creds\"" Oct 30 18:59:43.331455 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.331336 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"kube-root-ca.crt\"" Oct 30 18:59:43.331455 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.331423 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"openshift-service-ca.crt\"" Oct 30 18:59:43.331530 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.331488 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"default-dockercfg-zc4kv\"" Oct 30 18:59:43.333941 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.333921 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g"] Oct 30 18:59:43.381438 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.381407 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp248\" (UniqueName: \"kubernetes.io/projected/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-kube-api-access-tp248\") pod \"isvc-primary-29ca4c-predictor-858dbb99c6-7487g\" (UID: \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\") " pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 18:59:43.381438 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.381437 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-kube-rbac-proxy-sar-config\") pod \"isvc-primary-29ca4c-predictor-858dbb99c6-7487g\" (UID: \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\") " pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 18:59:43.381635 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.381490 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-kserve-provision-location\") pod \"isvc-primary-29ca4c-predictor-858dbb99c6-7487g\" (UID: \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\") " pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 18:59:43.381635 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.381508 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-cabundle-cert\") pod \"isvc-primary-29ca4c-predictor-858dbb99c6-7487g\" (UID: \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\") " pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 18:59:43.381635 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.381528 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-proxy-tls\") pod \"isvc-primary-29ca4c-predictor-858dbb99c6-7487g\" (UID: \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\") " pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 18:59:43.382529 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.381934 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-kserve-provision-location\") pod \"isvc-primary-29ca4c-predictor-858dbb99c6-7487g\" (UID: \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\") " pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 18:59:43.382529 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.382305 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-kube-rbac-proxy-sar-config\") pod \"isvc-primary-29ca4c-predictor-858dbb99c6-7487g\" (UID: \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\") " pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 18:59:43.382529 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.382484 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-cabundle-cert\") pod \"isvc-primary-29ca4c-predictor-858dbb99c6-7487g\" (UID: \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\") " pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 18:59:43.383760 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.383734 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-proxy-tls\") pod \"isvc-primary-29ca4c-predictor-858dbb99c6-7487g\" (UID: \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\") " pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 18:59:43.443077 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.443054 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp248\" (UniqueName: \"kubernetes.io/projected/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-kube-api-access-tp248\") pod \"isvc-primary-29ca4c-predictor-858dbb99c6-7487g\" (UID: \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\") " pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 18:59:43.546612 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.546540 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 18:59:43.972765 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:43.972733 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g"] Oct 30 18:59:43.978547 ip-10-0-132-74 kubenswrapper[5054]: W1030 18:59:43.978519 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30528265_6cfc_4aef_9ed1_e22bc8dd1dd2.slice/crio-7697726719f14fbfef4b71dca0549dae0885f95bee531b55ababef1ab13b23d9 WatchSource:0}: Error finding container 7697726719f14fbfef4b71dca0549dae0885f95bee531b55ababef1ab13b23d9: Status 404 returned error can't find the container with id 7697726719f14fbfef4b71dca0549dae0885f95bee531b55ababef1ab13b23d9 Oct 30 18:59:44.989221 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:44.989186 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" event={"ID":"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2","Type":"ContainerStarted","Data":"eaaa67804775860f26cd6871bfd05203400c5ed2c5c6ced20197bc338ceb4074"} Oct 30 18:59:44.989590 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:44.989265 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" event={"ID":"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2","Type":"ContainerStarted","Data":"7697726719f14fbfef4b71dca0549dae0885f95bee531b55ababef1ab13b23d9"} Oct 30 18:59:48.002665 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:48.002587 5054 generic.go:356] "Generic (PLEG): container finished" podID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerID="eaaa67804775860f26cd6871bfd05203400c5ed2c5c6ced20197bc338ceb4074" exitCode=0 Oct 30 18:59:48.003036 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:48.002664 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" event={"ID":"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2","Type":"ContainerDied","Data":"eaaa67804775860f26cd6871bfd05203400c5ed2c5c6ced20197bc338ceb4074"} Oct 30 18:59:49.009709 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:49.009673 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" event={"ID":"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2","Type":"ContainerStarted","Data":"7189a5ec61edbf3a633ad3e01873484cb2bee54f769baa3d13090bcfcbd471fb"} Oct 30 18:59:49.010211 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:49.009747 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" event={"ID":"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2","Type":"ContainerStarted","Data":"6fad54d1bb0ed507745371c4ecb542c425e8c8f25e8548b1aceafcaa2dc0a852"} Oct 30 18:59:49.010211 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:49.009969 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 18:59:49.129538 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:49.129494 5054 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" podStartSLOduration=2.458645875 podStartE2EDuration="6.129480917s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.670835042s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:49.127698135 +0000 UTC m=+1330.856076850" watchObservedRunningTime="2026-10-30 18:59:49.129480917 +0000 UTC m=+1330.857859632" Oct 30 18:59:50.013944 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:50.013907 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 18:59:50.015083 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:50.015058 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused" Oct 30 18:59:51.017928 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:51.017893 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused" Oct 30 18:59:56.022333 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:56.022304 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 18:59:56.022771 ip-10-0-132-74 kubenswrapper[5054]: I1030 18:59:56.022746 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused" Oct 30 19:00:06.023124 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:00:06.023084 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused" Oct 30 19:00:08.595974 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:00:08.595936 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:00:16.023180 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:00:16.023137 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused" Oct 30 19:00:26.022944 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:00:26.022901 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused" Oct 30 19:00:36.023226 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:00:36.023183 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused" Oct 30 19:00:38.596524 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:00:38.596488 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:00:46.022723 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:00:46.022681 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused" Oct 30 19:00:56.023184 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:00:56.023149 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused" Oct 30 19:01:03.417266 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:03.417231 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused" Oct 30 19:01:08.597130 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:01:08.597094 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:01:13.418658 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:13.418631 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 19:01:38.596455 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:01:38.596422 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:01:38.728937 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:38.728905 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7"] Oct 30 19:01:38.732597 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:38.732576 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" Oct 30 19:01:38.832597 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:38.832568 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"isvc-secondary-29ca4c-kube-rbac-proxy-sar-config\"" Oct 30 19:01:38.832597 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:38.832575 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"invalid-s3-secret-29ca4c\"" Oct 30 19:01:38.832786 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:38.832597 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"isvc-secondary-29ca4c-predictor-serving-cert\"" Oct 30 19:01:38.832786 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:38.832578 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"invalid-s3-sa-29ca4c-dockercfg-ddz6k\"" Oct 30 19:01:38.833971 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:38.833954 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7"] Oct 30 19:01:38.875243 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:38.875215 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b01bc531-bc86-4092-8498-d59a6f5e2b43-cabundle-cert\") pod \"isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7\" (UID: \"b01bc531-bc86-4092-8498-d59a6f5e2b43\") " pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" Oct 30 19:01:38.875391 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:38.875265 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzt8k\" (UniqueName: \"kubernetes.io/projected/b01bc531-bc86-4092-8498-d59a6f5e2b43-kube-api-access-nzt8k\") pod \"isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7\" (UID: \"b01bc531-bc86-4092-8498-d59a6f5e2b43\") " pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" Oct 30 19:01:38.875391 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:38.875289 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b01bc531-bc86-4092-8498-d59a6f5e2b43-kube-rbac-proxy-sar-config\") pod \"isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7\" (UID: \"b01bc531-bc86-4092-8498-d59a6f5e2b43\") " pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" Oct 30 19:01:38.875476 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:38.875415 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b01bc531-bc86-4092-8498-d59a6f5e2b43-kserve-provision-location\") pod \"isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7\" (UID: \"b01bc531-bc86-4092-8498-d59a6f5e2b43\") " pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" Oct 30 19:01:38.875476 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:38.875462 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b01bc531-bc86-4092-8498-d59a6f5e2b43-proxy-tls\") pod \"isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7\" (UID: \"b01bc531-bc86-4092-8498-d59a6f5e2b43\") " pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" Oct 30 19:01:38.976514 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:38.976487 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzt8k\" (UniqueName: \"kubernetes.io/projected/b01bc531-bc86-4092-8498-d59a6f5e2b43-kube-api-access-nzt8k\") pod \"isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7\" (UID: \"b01bc531-bc86-4092-8498-d59a6f5e2b43\") " pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" Oct 30 19:01:38.976669 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:38.976518 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b01bc531-bc86-4092-8498-d59a6f5e2b43-kube-rbac-proxy-sar-config\") pod \"isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7\" (UID: \"b01bc531-bc86-4092-8498-d59a6f5e2b43\") " pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" Oct 30 19:01:38.976669 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:38.976539 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b01bc531-bc86-4092-8498-d59a6f5e2b43-kserve-provision-location\") pod \"isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7\" (UID: \"b01bc531-bc86-4092-8498-d59a6f5e2b43\") " pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" Oct 30 19:01:38.976669 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:38.976561 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b01bc531-bc86-4092-8498-d59a6f5e2b43-proxy-tls\") pod \"isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7\" (UID: \"b01bc531-bc86-4092-8498-d59a6f5e2b43\") " pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" Oct 30 19:01:38.976669 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:38.976613 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b01bc531-bc86-4092-8498-d59a6f5e2b43-cabundle-cert\") pod \"isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7\" (UID: \"b01bc531-bc86-4092-8498-d59a6f5e2b43\") " pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" Oct 30 19:01:38.976988 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:38.976964 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b01bc531-bc86-4092-8498-d59a6f5e2b43-kserve-provision-location\") pod \"isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7\" (UID: \"b01bc531-bc86-4092-8498-d59a6f5e2b43\") " pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" Oct 30 19:01:38.977315 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:38.977293 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b01bc531-bc86-4092-8498-d59a6f5e2b43-kube-rbac-proxy-sar-config\") pod \"isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7\" (UID: \"b01bc531-bc86-4092-8498-d59a6f5e2b43\") " pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" Oct 30 19:01:38.977385 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:38.977331 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b01bc531-bc86-4092-8498-d59a6f5e2b43-cabundle-cert\") pod \"isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7\" (UID: \"b01bc531-bc86-4092-8498-d59a6f5e2b43\") " pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" Oct 30 19:01:38.978988 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:38.978973 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b01bc531-bc86-4092-8498-d59a6f5e2b43-proxy-tls\") pod \"isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7\" (UID: \"b01bc531-bc86-4092-8498-d59a6f5e2b43\") " pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" Oct 30 19:01:39.038075 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:39.038052 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzt8k\" (UniqueName: \"kubernetes.io/projected/b01bc531-bc86-4092-8498-d59a6f5e2b43-kube-api-access-nzt8k\") pod \"isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7\" (UID: \"b01bc531-bc86-4092-8498-d59a6f5e2b43\") " pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" Oct 30 19:01:39.049892 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:39.049872 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" Oct 30 19:01:39.472134 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:39.472101 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7"] Oct 30 19:01:39.478229 ip-10-0-132-74 kubenswrapper[5054]: W1030 19:01:39.478202 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb01bc531_bc86_4092_8498_d59a6f5e2b43.slice/crio-a2ecd0c4ea9e6c031f4d70ce12d723e357c6fe488216e212c7448f385a52572e WatchSource:0}: Error finding container a2ecd0c4ea9e6c031f4d70ce12d723e357c6fe488216e212c7448f385a52572e: Status 404 returned error can't find the container with id a2ecd0c4ea9e6c031f4d70ce12d723e357c6fe488216e212c7448f385a52572e Oct 30 19:01:40.467370 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:40.467328 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" event={"ID":"b01bc531-bc86-4092-8498-d59a6f5e2b43","Type":"ContainerStarted","Data":"20abb7bfb9d2700547ac2a9780623d17f47ea8f69d2d3fe8d6e28e12dd6f5892"} Oct 30 19:01:40.467742 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:40.467401 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" event={"ID":"b01bc531-bc86-4092-8498-d59a6f5e2b43","Type":"ContainerStarted","Data":"a2ecd0c4ea9e6c031f4d70ce12d723e357c6fe488216e212c7448f385a52572e"} Oct 30 19:01:44.486991 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:44.486962 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-e43a086b9d_isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7_b01bc531-bc86-4092-8498-d59a6f5e2b43/storage-initializer/0.log" Oct 30 19:01:44.487375 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:44.487018 5054 generic.go:356] "Generic (PLEG): container finished" podID="b01bc531-bc86-4092-8498-d59a6f5e2b43" containerID="20abb7bfb9d2700547ac2a9780623d17f47ea8f69d2d3fe8d6e28e12dd6f5892" exitCode=1 Oct 30 19:01:44.487375 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:44.487031 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" event={"ID":"b01bc531-bc86-4092-8498-d59a6f5e2b43","Type":"ContainerDied","Data":"20abb7bfb9d2700547ac2a9780623d17f47ea8f69d2d3fe8d6e28e12dd6f5892"} Oct 30 19:01:45.493273 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:45.493245 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-e43a086b9d_isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7_b01bc531-bc86-4092-8498-d59a6f5e2b43/storage-initializer/0.log" Oct 30 19:01:45.493650 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:45.493322 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" event={"ID":"b01bc531-bc86-4092-8498-d59a6f5e2b43","Type":"ContainerStarted","Data":"ae2e9696a217a50ca0f7140bec44eb457b5d7037374bc65dadff407771a14b10"} Oct 30 19:01:48.505481 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:48.505453 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-e43a086b9d_isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7_b01bc531-bc86-4092-8498-d59a6f5e2b43/storage-initializer/1.log" Oct 30 19:01:48.505852 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:48.505827 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-e43a086b9d_isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7_b01bc531-bc86-4092-8498-d59a6f5e2b43/storage-initializer/0.log" Oct 30 19:01:48.505899 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:48.505858 5054 generic.go:356] "Generic (PLEG): container finished" podID="b01bc531-bc86-4092-8498-d59a6f5e2b43" containerID="ae2e9696a217a50ca0f7140bec44eb457b5d7037374bc65dadff407771a14b10" exitCode=1 Oct 30 19:01:48.505942 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:48.505903 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" event={"ID":"b01bc531-bc86-4092-8498-d59a6f5e2b43","Type":"ContainerDied","Data":"ae2e9696a217a50ca0f7140bec44eb457b5d7037374bc65dadff407771a14b10"} Oct 30 19:01:48.506017 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:48.505953 5054 scope.go:122] "RemoveContainer" containerID="20abb7bfb9d2700547ac2a9780623d17f47ea8f69d2d3fe8d6e28e12dd6f5892" Oct 30 19:01:48.506306 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:48.506282 5054 scope.go:122] "RemoveContainer" containerID="20abb7bfb9d2700547ac2a9780623d17f47ea8f69d2d3fe8d6e28e12dd6f5892" Oct 30 19:01:48.516795 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:01:48.516769 5054 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"storage-initializer\" with CrashLoopBackOff: \"back-off 10s restarting failed container=storage-initializer pod=isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7_e2e-gw1-e43a086b9d(b01bc531-bc86-4092-8498-d59a6f5e2b43)\"" pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" podUID="b01bc531-bc86-4092-8498-d59a6f5e2b43" Oct 30 19:01:48.517064 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:01:48.517035 5054 log.go:32] "RemoveContainer from runtime service failed" err="rpc error: code = Unknown desc = failed to delete container k8s_storage-initializer_isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7_e2e-gw1-e43a086b9d_b01bc531-bc86-4092-8498-d59a6f5e2b43_0 in pod sandbox a2ecd0c4ea9e6c031f4d70ce12d723e357c6fe488216e212c7448f385a52572e from index: no such id: '20abb7bfb9d2700547ac2a9780623d17f47ea8f69d2d3fe8d6e28e12dd6f5892'" containerID="20abb7bfb9d2700547ac2a9780623d17f47ea8f69d2d3fe8d6e28e12dd6f5892" Oct 30 19:01:48.517157 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:48.517070 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20abb7bfb9d2700547ac2a9780623d17f47ea8f69d2d3fe8d6e28e12dd6f5892"} err="rpc error: code = Unknown desc = failed to delete container k8s_storage-initializer_isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7_e2e-gw1-e43a086b9d_b01bc531-bc86-4092-8498-d59a6f5e2b43_0 in pod sandbox a2ecd0c4ea9e6c031f4d70ce12d723e357c6fe488216e212c7448f385a52572e from index: no such id: '20abb7bfb9d2700547ac2a9780623d17f47ea8f69d2d3fe8d6e28e12dd6f5892'" Oct 30 19:01:49.511445 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:49.511416 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-e43a086b9d_isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7_b01bc531-bc86-4092-8498-d59a6f5e2b43/storage-initializer/1.log" Oct 30 19:01:58.693832 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:58.693796 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7"] Oct 30 19:01:58.830090 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:58.830067 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-e43a086b9d_isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7_b01bc531-bc86-4092-8498-d59a6f5e2b43/storage-initializer/1.log" Oct 30 19:01:58.832452 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:58.832436 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" Oct 30 19:01:58.950886 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:58.950807 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b01bc531-bc86-4092-8498-d59a6f5e2b43-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b01bc531-bc86-4092-8498-d59a6f5e2b43-kserve-provision-location\") pod \"b01bc531-bc86-4092-8498-d59a6f5e2b43\" (UID: \"b01bc531-bc86-4092-8498-d59a6f5e2b43\") " Oct 30 19:01:58.950886 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:58.950862 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b01bc531-bc86-4092-8498-d59a6f5e2b43-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b01bc531-bc86-4092-8498-d59a6f5e2b43-kube-rbac-proxy-sar-config\") pod \"b01bc531-bc86-4092-8498-d59a6f5e2b43\" (UID: \"b01bc531-bc86-4092-8498-d59a6f5e2b43\") " Oct 30 19:01:58.951186 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:58.950937 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b01bc531-bc86-4092-8498-d59a6f5e2b43-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b01bc531-bc86-4092-8498-d59a6f5e2b43-cabundle-cert\") pod \"b01bc531-bc86-4092-8498-d59a6f5e2b43\" (UID: \"b01bc531-bc86-4092-8498-d59a6f5e2b43\") " Oct 30 19:01:58.951186 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:58.950974 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b01bc531-bc86-4092-8498-d59a6f5e2b43-kube-api-access-nzt8k\" (UniqueName: \"kubernetes.io/projected/b01bc531-bc86-4092-8498-d59a6f5e2b43-kube-api-access-nzt8k\") pod \"b01bc531-bc86-4092-8498-d59a6f5e2b43\" (UID: \"b01bc531-bc86-4092-8498-d59a6f5e2b43\") " Oct 30 19:01:58.951186 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:58.951043 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b01bc531-bc86-4092-8498-d59a6f5e2b43-proxy-tls\" (UniqueName: \"kubernetes.io/secret/b01bc531-bc86-4092-8498-d59a6f5e2b43-proxy-tls\") pod \"b01bc531-bc86-4092-8498-d59a6f5e2b43\" (UID: \"b01bc531-bc86-4092-8498-d59a6f5e2b43\") " Oct 30 19:01:58.951186 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:58.951157 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b01bc531-bc86-4092-8498-d59a6f5e2b43-kserve-provision-location" pod "b01bc531-bc86-4092-8498-d59a6f5e2b43" (UID: "b01bc531-bc86-4092-8498-d59a6f5e2b43"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:01:58.951766 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:58.951723 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b01bc531-bc86-4092-8498-d59a6f5e2b43-cabundle-cert" pod "b01bc531-bc86-4092-8498-d59a6f5e2b43" (UID: "b01bc531-bc86-4092-8498-d59a6f5e2b43"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:01:58.952116 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:58.952080 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b01bc531-bc86-4092-8498-d59a6f5e2b43-kube-rbac-proxy-sar-config" pod "b01bc531-bc86-4092-8498-d59a6f5e2b43" (UID: "b01bc531-bc86-4092-8498-d59a6f5e2b43"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:01:58.952482 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:58.952462 5054 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b01bc531-bc86-4092-8498-d59a6f5e2b43-kserve-provision-location\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:58.952663 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:58.952638 5054 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b01bc531-bc86-4092-8498-d59a6f5e2b43-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:58.952663 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:58.952661 5054 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b01bc531-bc86-4092-8498-d59a6f5e2b43-cabundle-cert\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:58.953712 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:58.953687 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b01bc531-bc86-4092-8498-d59a6f5e2b43-proxy-tls" pod "b01bc531-bc86-4092-8498-d59a6f5e2b43" (UID: "b01bc531-bc86-4092-8498-d59a6f5e2b43"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:01:58.955378 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:58.955355 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b01bc531-bc86-4092-8498-d59a6f5e2b43-kube-api-access-nzt8k" pod "b01bc531-bc86-4092-8498-d59a6f5e2b43" (UID: "b01bc531-bc86-4092-8498-d59a6f5e2b43"). InnerVolumeSpecName "kube-api-access-nzt8k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:01:58.966934 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:58.966911 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g"] Oct 30 19:01:58.967167 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:58.967144 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerName="kserve-container" containerID="cri-o://6fad54d1bb0ed507745371c4ecb542c425e8c8f25e8548b1aceafcaa2dc0a852" gracePeriod=30 Oct 30 19:01:58.967231 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:58.967188 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerName="kube-rbac-proxy" containerID="cri-o://7189a5ec61edbf3a633ad3e01873484cb2bee54f769baa3d13090bcfcbd471fb" gracePeriod=30 Oct 30 19:01:59.053421 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:59.053395 5054 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzt8k\" (UniqueName: \"kubernetes.io/projected/b01bc531-bc86-4092-8498-d59a6f5e2b43-kube-api-access-nzt8k\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:59.053421 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:59.053418 5054 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b01bc531-bc86-4092-8498-d59a6f5e2b43-proxy-tls\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:01:59.550157 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:59.550127 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-e43a086b9d_isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7_b01bc531-bc86-4092-8498-d59a6f5e2b43/storage-initializer/1.log" Oct 30 19:01:59.550341 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:59.550239 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" event={"ID":"b01bc531-bc86-4092-8498-d59a6f5e2b43","Type":"ContainerDied","Data":"a2ecd0c4ea9e6c031f4d70ce12d723e357c6fe488216e212c7448f385a52572e"} Oct 30 19:01:59.550341 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:59.550243 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7" Oct 30 19:01:59.550462 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:59.550290 5054 scope.go:122] "RemoveContainer" containerID="ae2e9696a217a50ca0f7140bec44eb457b5d7037374bc65dadff407771a14b10" Oct 30 19:01:59.552630 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:59.552609 5054 generic.go:356] "Generic (PLEG): container finished" podID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerID="7189a5ec61edbf3a633ad3e01873484cb2bee54f769baa3d13090bcfcbd471fb" exitCode=0 Oct 30 19:01:59.552717 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:59.552659 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" event={"ID":"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2","Type":"ContainerDied","Data":"7189a5ec61edbf3a633ad3e01873484cb2bee54f769baa3d13090bcfcbd471fb"} Oct 30 19:01:59.779391 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:59.779357 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7"] Oct 30 19:01:59.785515 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:01:59.785492 5054 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-secondary-29ca4c-predictor-89754c8c5-lrkb7"] Oct 30 19:02:00.421469 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:00.421438 5054 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b01bc531-bc86-4092-8498-d59a6f5e2b43" path="/var/lib/kubelet/pods/b01bc531-bc86-4092-8498-d59a6f5e2b43/volumes" Oct 30 19:02:01.019050 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:01.018988 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.41:8643/healthz\": dial tcp 10.132.0.41:8643: connect: connection refused" Oct 30 19:02:03.417918 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:03.417879 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.41:8080: connect: connection refused" Oct 30 19:02:03.712479 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:03.712455 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 19:02:03.792335 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:03.792311 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-kube-rbac-proxy-sar-config\") pod \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\" (UID: \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\") " Oct 30 19:02:03.792454 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:03.792344 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-kserve-provision-location\") pod \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\" (UID: \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\") " Oct 30 19:02:03.792454 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:03.792408 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-proxy-tls\" (UniqueName: \"kubernetes.io/secret/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-proxy-tls\") pod \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\" (UID: \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\") " Oct 30 19:02:03.792454 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:03.792438 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-kube-api-access-tp248\" (UniqueName: \"kubernetes.io/projected/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-kube-api-access-tp248\") pod \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\" (UID: \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\") " Oct 30 19:02:03.792576 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:03.792482 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-cabundle-cert\") pod \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\" (UID: \"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2\") " Oct 30 19:02:03.792737 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:03.792708 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-kserve-provision-location" pod "30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" (UID: "30528265-6cfc-4aef-9ed1-e22bc8dd1dd2"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:02:03.792815 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:03.792721 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-kube-rbac-proxy-sar-config" pod "30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" (UID: "30528265-6cfc-4aef-9ed1-e22bc8dd1dd2"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:02:03.792935 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:03.792912 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-cabundle-cert" pod "30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" (UID: "30528265-6cfc-4aef-9ed1-e22bc8dd1dd2"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:02:03.794511 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:03.794487 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-proxy-tls" pod "30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" (UID: "30528265-6cfc-4aef-9ed1-e22bc8dd1dd2"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:02:03.794597 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:03.794554 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-kube-api-access-tp248" pod "30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" (UID: "30528265-6cfc-4aef-9ed1-e22bc8dd1dd2"). InnerVolumeSpecName "kube-api-access-tp248". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:02:03.893095 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:03.893072 5054 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:02:03.893180 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:03.893100 5054 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-kserve-provision-location\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:02:03.893180 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:03.893111 5054 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-proxy-tls\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:02:03.893180 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:03.893119 5054 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tp248\" (UniqueName: \"kubernetes.io/projected/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-kube-api-access-tp248\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:02:03.893180 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:03.893127 5054 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2-cabundle-cert\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:02:04.574746 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:04.574712 5054 generic.go:356] "Generic (PLEG): container finished" podID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerID="6fad54d1bb0ed507745371c4ecb542c425e8c8f25e8548b1aceafcaa2dc0a852" exitCode=0 Oct 30 19:02:04.575164 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:04.574767 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" Oct 30 19:02:04.575164 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:04.574801 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" event={"ID":"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2","Type":"ContainerDied","Data":"6fad54d1bb0ed507745371c4ecb542c425e8c8f25e8548b1aceafcaa2dc0a852"} Oct 30 19:02:04.575164 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:04.574856 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g" event={"ID":"30528265-6cfc-4aef-9ed1-e22bc8dd1dd2","Type":"ContainerDied","Data":"7697726719f14fbfef4b71dca0549dae0885f95bee531b55ababef1ab13b23d9"} Oct 30 19:02:04.575164 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:04.574884 5054 scope.go:122] "RemoveContainer" containerID="7189a5ec61edbf3a633ad3e01873484cb2bee54f769baa3d13090bcfcbd471fb" Oct 30 19:02:04.583069 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:04.583052 5054 scope.go:122] "RemoveContainer" containerID="6fad54d1bb0ed507745371c4ecb542c425e8c8f25e8548b1aceafcaa2dc0a852" Oct 30 19:02:04.590185 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:04.590028 5054 scope.go:122] "RemoveContainer" containerID="eaaa67804775860f26cd6871bfd05203400c5ed2c5c6ced20197bc338ceb4074" Oct 30 19:02:04.596738 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:04.596723 5054 scope.go:122] "RemoveContainer" containerID="7189a5ec61edbf3a633ad3e01873484cb2bee54f769baa3d13090bcfcbd471fb" Oct 30 19:02:04.596941 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:02:04.596921 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7189a5ec61edbf3a633ad3e01873484cb2bee54f769baa3d13090bcfcbd471fb\": container with ID starting with 7189a5ec61edbf3a633ad3e01873484cb2bee54f769baa3d13090bcfcbd471fb not found: ID does not exist" containerID="7189a5ec61edbf3a633ad3e01873484cb2bee54f769baa3d13090bcfcbd471fb" Oct 30 19:02:04.596996 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:04.596949 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7189a5ec61edbf3a633ad3e01873484cb2bee54f769baa3d13090bcfcbd471fb"} err="failed to get container status \"7189a5ec61edbf3a633ad3e01873484cb2bee54f769baa3d13090bcfcbd471fb\": rpc error: code = NotFound desc = could not find container \"7189a5ec61edbf3a633ad3e01873484cb2bee54f769baa3d13090bcfcbd471fb\": container with ID starting with 7189a5ec61edbf3a633ad3e01873484cb2bee54f769baa3d13090bcfcbd471fb not found: ID does not exist" Oct 30 19:02:04.596996 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:04.596966 5054 scope.go:122] "RemoveContainer" containerID="6fad54d1bb0ed507745371c4ecb542c425e8c8f25e8548b1aceafcaa2dc0a852" Oct 30 19:02:04.597236 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:02:04.597216 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fad54d1bb0ed507745371c4ecb542c425e8c8f25e8548b1aceafcaa2dc0a852\": container with ID starting with 6fad54d1bb0ed507745371c4ecb542c425e8c8f25e8548b1aceafcaa2dc0a852 not found: ID does not exist" containerID="6fad54d1bb0ed507745371c4ecb542c425e8c8f25e8548b1aceafcaa2dc0a852" Oct 30 19:02:04.597278 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:04.597242 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fad54d1bb0ed507745371c4ecb542c425e8c8f25e8548b1aceafcaa2dc0a852"} err="failed to get container status \"6fad54d1bb0ed507745371c4ecb542c425e8c8f25e8548b1aceafcaa2dc0a852\": rpc error: code = NotFound desc = could not find container \"6fad54d1bb0ed507745371c4ecb542c425e8c8f25e8548b1aceafcaa2dc0a852\": container with ID starting with 6fad54d1bb0ed507745371c4ecb542c425e8c8f25e8548b1aceafcaa2dc0a852 not found: ID does not exist" Oct 30 19:02:04.597278 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:04.597256 5054 scope.go:122] "RemoveContainer" containerID="eaaa67804775860f26cd6871bfd05203400c5ed2c5c6ced20197bc338ceb4074" Oct 30 19:02:04.597478 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:02:04.597462 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaaa67804775860f26cd6871bfd05203400c5ed2c5c6ced20197bc338ceb4074\": container with ID starting with eaaa67804775860f26cd6871bfd05203400c5ed2c5c6ced20197bc338ceb4074 not found: ID does not exist" containerID="eaaa67804775860f26cd6871bfd05203400c5ed2c5c6ced20197bc338ceb4074" Oct 30 19:02:04.597518 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:04.597482 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaaa67804775860f26cd6871bfd05203400c5ed2c5c6ced20197bc338ceb4074"} err="failed to get container status \"eaaa67804775860f26cd6871bfd05203400c5ed2c5c6ced20197bc338ceb4074\": rpc error: code = NotFound desc = could not find container \"eaaa67804775860f26cd6871bfd05203400c5ed2c5c6ced20197bc338ceb4074\": container with ID starting with eaaa67804775860f26cd6871bfd05203400c5ed2c5c6ced20197bc338ceb4074 not found: ID does not exist" Oct 30 19:02:04.784318 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:04.784242 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g"] Oct 30 19:02:04.832195 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:04.832161 5054 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-primary-29ca4c-predictor-858dbb99c6-7487g"] Oct 30 19:02:06.421249 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:06.421216 5054 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" path="/var/lib/kubelet/pods/30528265-6cfc-4aef-9ed1-e22bc8dd1dd2/volumes" Oct 30 19:02:07.886769 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.886734 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8"] Oct 30 19:02:07.887154 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.887100 5054 cpu_manager.go:414] "containerMap: removing container" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerName="kube-rbac-proxy" Oct 30 19:02:07.887154 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.887114 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerName="kube-rbac-proxy" Oct 30 19:02:07.887154 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.887130 5054 cpu_manager.go:414] "containerMap: removing container" podUID="b01bc531-bc86-4092-8498-d59a6f5e2b43" containerName="storage-initializer" Oct 30 19:02:07.887154 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.887136 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b01bc531-bc86-4092-8498-d59a6f5e2b43" containerName="storage-initializer" Oct 30 19:02:07.887154 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.887143 5054 cpu_manager.go:414] "containerMap: removing container" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerName="storage-initializer" Oct 30 19:02:07.887154 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.887149 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerName="storage-initializer" Oct 30 19:02:07.887154 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.887154 5054 cpu_manager.go:414] "containerMap: removing container" podUID="b01bc531-bc86-4092-8498-d59a6f5e2b43" containerName="storage-initializer" Oct 30 19:02:07.887154 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.887159 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b01bc531-bc86-4092-8498-d59a6f5e2b43" containerName="storage-initializer" Oct 30 19:02:07.887404 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.887167 5054 cpu_manager.go:414] "containerMap: removing container" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerName="kserve-container" Oct 30 19:02:07.887404 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.887172 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerName="kserve-container" Oct 30 19:02:07.887404 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.887231 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerName="kserve-container" Oct 30 19:02:07.887404 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.887244 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="30528265-6cfc-4aef-9ed1-e22bc8dd1dd2" containerName="kube-rbac-proxy" Oct 30 19:02:07.887404 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.887251 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="b01bc531-bc86-4092-8498-d59a6f5e2b43" containerName="storage-initializer" Oct 30 19:02:07.887404 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.887257 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="b01bc531-bc86-4092-8498-d59a6f5e2b43" containerName="storage-initializer" Oct 30 19:02:07.891933 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.891915 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" Oct 30 19:02:07.926502 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.926476 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/25a92414-e34a-4711-b877-5decf9086277-kube-rbac-proxy-sar-config\") pod \"isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8\" (UID: \"25a92414-e34a-4711-b877-5decf9086277\") " pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" Oct 30 19:02:07.926635 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.926509 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/25a92414-e34a-4711-b877-5decf9086277-kserve-provision-location\") pod \"isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8\" (UID: \"25a92414-e34a-4711-b877-5decf9086277\") " pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" Oct 30 19:02:07.926635 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.926551 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/25a92414-e34a-4711-b877-5decf9086277-proxy-tls\") pod \"isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8\" (UID: \"25a92414-e34a-4711-b877-5decf9086277\") " pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" Oct 30 19:02:07.926717 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.926642 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/25a92414-e34a-4711-b877-5decf9086277-cabundle-cert\") pod \"isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8\" (UID: \"25a92414-e34a-4711-b877-5decf9086277\") " pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" Oct 30 19:02:07.926717 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.926664 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z62v9\" (UniqueName: \"kubernetes.io/projected/25a92414-e34a-4711-b877-5decf9086277-kube-api-access-z62v9\") pod \"isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8\" (UID: \"25a92414-e34a-4711-b877-5decf9086277\") " pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" Oct 30 19:02:07.994532 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.994503 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"fail-s3-secret-95186e\"" Oct 30 19:02:07.994674 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.994536 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"odh-kserve-custom-ca-bundle\"" Oct 30 19:02:07.994674 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.994543 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"isvc-init-fail-95186e-kube-rbac-proxy-sar-config\"" Oct 30 19:02:07.994674 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.994561 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"openshift-service-ca.crt\"" Oct 30 19:02:07.994674 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.994547 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"fail-s3-sa-95186e-dockercfg-6mzjf\"" Oct 30 19:02:07.994674 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.994565 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"kube-root-ca.crt\"" Oct 30 19:02:07.994674 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.994604 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"isvc-init-fail-95186e-predictor-serving-cert\"" Oct 30 19:02:07.995813 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:07.995792 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8"] Oct 30 19:02:08.027637 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:08.027614 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/25a92414-e34a-4711-b877-5decf9086277-proxy-tls\") pod \"isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8\" (UID: \"25a92414-e34a-4711-b877-5decf9086277\") " pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" Oct 30 19:02:08.027777 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:08.027648 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/25a92414-e34a-4711-b877-5decf9086277-cabundle-cert\") pod \"isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8\" (UID: \"25a92414-e34a-4711-b877-5decf9086277\") " pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" Oct 30 19:02:08.027777 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:08.027672 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z62v9\" (UniqueName: \"kubernetes.io/projected/25a92414-e34a-4711-b877-5decf9086277-kube-api-access-z62v9\") pod \"isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8\" (UID: \"25a92414-e34a-4711-b877-5decf9086277\") " pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" Oct 30 19:02:08.027777 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:08.027705 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/25a92414-e34a-4711-b877-5decf9086277-kube-rbac-proxy-sar-config\") pod \"isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8\" (UID: \"25a92414-e34a-4711-b877-5decf9086277\") " pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" Oct 30 19:02:08.027777 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:08.027728 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/25a92414-e34a-4711-b877-5decf9086277-kserve-provision-location\") pod \"isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8\" (UID: \"25a92414-e34a-4711-b877-5decf9086277\") " pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" Oct 30 19:02:08.028198 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:08.028169 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/25a92414-e34a-4711-b877-5decf9086277-kserve-provision-location\") pod \"isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8\" (UID: \"25a92414-e34a-4711-b877-5decf9086277\") " pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" Oct 30 19:02:08.028412 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:08.028395 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/25a92414-e34a-4711-b877-5decf9086277-kube-rbac-proxy-sar-config\") pod \"isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8\" (UID: \"25a92414-e34a-4711-b877-5decf9086277\") " pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" Oct 30 19:02:08.028461 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:08.028406 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/25a92414-e34a-4711-b877-5decf9086277-cabundle-cert\") pod \"isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8\" (UID: \"25a92414-e34a-4711-b877-5decf9086277\") " pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" Oct 30 19:02:08.030057 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:08.030035 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/25a92414-e34a-4711-b877-5decf9086277-proxy-tls\") pod \"isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8\" (UID: \"25a92414-e34a-4711-b877-5decf9086277\") " pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" Oct 30 19:02:08.088868 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:08.088832 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z62v9\" (UniqueName: \"kubernetes.io/projected/25a92414-e34a-4711-b877-5decf9086277-kube-api-access-z62v9\") pod \"isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8\" (UID: \"25a92414-e34a-4711-b877-5decf9086277\") " pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" Oct 30 19:02:08.208936 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:08.208861 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" Oct 30 19:02:08.597565 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:02:08.597476 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:02:08.630298 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:08.630270 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8"] Oct 30 19:02:08.635786 ip-10-0-132-74 kubenswrapper[5054]: W1030 19:02:08.635760 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25a92414_e34a_4711_b877_5decf9086277.slice/crio-016f10071f85b2b455f39b6376a5441a0dd8f3e9beb2943b331a95a25eb17ccd WatchSource:0}: Error finding container 016f10071f85b2b455f39b6376a5441a0dd8f3e9beb2943b331a95a25eb17ccd: Status 404 returned error can't find the container with id 016f10071f85b2b455f39b6376a5441a0dd8f3e9beb2943b331a95a25eb17ccd Oct 30 19:02:09.599511 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:09.599476 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" event={"ID":"25a92414-e34a-4711-b877-5decf9086277","Type":"ContainerStarted","Data":"6a17f6b025e3a4158d6ffbdd51d1863f2789496e27de735af097ea5f662431fd"} Oct 30 19:02:09.599998 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:09.599542 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" event={"ID":"25a92414-e34a-4711-b877-5decf9086277","Type":"ContainerStarted","Data":"016f10071f85b2b455f39b6376a5441a0dd8f3e9beb2943b331a95a25eb17ccd"} Oct 30 19:02:12.613153 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:12.613127 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-e43a086b9d_isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8_25a92414-e34a-4711-b877-5decf9086277/storage-initializer/0.log" Oct 30 19:02:12.613556 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:12.613162 5054 generic.go:356] "Generic (PLEG): container finished" podID="25a92414-e34a-4711-b877-5decf9086277" containerID="6a17f6b025e3a4158d6ffbdd51d1863f2789496e27de735af097ea5f662431fd" exitCode=1 Oct 30 19:02:12.613556 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:12.613199 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" event={"ID":"25a92414-e34a-4711-b877-5decf9086277","Type":"ContainerDied","Data":"6a17f6b025e3a4158d6ffbdd51d1863f2789496e27de735af097ea5f662431fd"} Oct 30 19:02:13.619372 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:13.619342 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-e43a086b9d_isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8_25a92414-e34a-4711-b877-5decf9086277/storage-initializer/0.log" Oct 30 19:02:13.619742 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:13.619428 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" event={"ID":"25a92414-e34a-4711-b877-5decf9086277","Type":"ContainerStarted","Data":"42849f100d508b3b5dd5dff7d794266166d0184e8d243b54d546bfb24728d3f8"} Oct 30 19:02:17.458525 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:17.458484 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8"] Oct 30 19:02:17.458986 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:17.458730 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" podUID="25a92414-e34a-4711-b877-5decf9086277" containerName="storage-initializer" containerID="cri-o://42849f100d508b3b5dd5dff7d794266166d0184e8d243b54d546bfb24728d3f8" gracePeriod=30 Oct 30 19:02:20.391723 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.391698 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-e43a086b9d_isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8_25a92414-e34a-4711-b877-5decf9086277/storage-initializer/1.log" Oct 30 19:02:20.392149 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.392131 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-e43a086b9d_isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8_25a92414-e34a-4711-b877-5decf9086277/storage-initializer/0.log" Oct 30 19:02:20.394667 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.394648 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" Oct 30 19:02:20.525862 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.525791 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/25a92414-e34a-4711-b877-5decf9086277-proxy-tls\" (UniqueName: \"kubernetes.io/secret/25a92414-e34a-4711-b877-5decf9086277-proxy-tls\") pod \"25a92414-e34a-4711-b877-5decf9086277\" (UID: \"25a92414-e34a-4711-b877-5decf9086277\") " Oct 30 19:02:20.525862 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.525829 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25a92414-e34a-4711-b877-5decf9086277-kube-api-access-z62v9\" (UniqueName: \"kubernetes.io/projected/25a92414-e34a-4711-b877-5decf9086277-kube-api-access-z62v9\") pod \"25a92414-e34a-4711-b877-5decf9086277\" (UID: \"25a92414-e34a-4711-b877-5decf9086277\") " Oct 30 19:02:20.525862 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.525857 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/25a92414-e34a-4711-b877-5decf9086277-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/25a92414-e34a-4711-b877-5decf9086277-kube-rbac-proxy-sar-config\") pod \"25a92414-e34a-4711-b877-5decf9086277\" (UID: \"25a92414-e34a-4711-b877-5decf9086277\") " Oct 30 19:02:20.526166 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.525878 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25a92414-e34a-4711-b877-5decf9086277-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/25a92414-e34a-4711-b877-5decf9086277-kserve-provision-location\") pod \"25a92414-e34a-4711-b877-5decf9086277\" (UID: \"25a92414-e34a-4711-b877-5decf9086277\") " Oct 30 19:02:20.526166 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.525897 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/25a92414-e34a-4711-b877-5decf9086277-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/25a92414-e34a-4711-b877-5decf9086277-cabundle-cert\") pod \"25a92414-e34a-4711-b877-5decf9086277\" (UID: \"25a92414-e34a-4711-b877-5decf9086277\") " Oct 30 19:02:20.526278 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.526199 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25a92414-e34a-4711-b877-5decf9086277-kserve-provision-location" pod "25a92414-e34a-4711-b877-5decf9086277" (UID: "25a92414-e34a-4711-b877-5decf9086277"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:02:20.526278 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.526241 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25a92414-e34a-4711-b877-5decf9086277-kube-rbac-proxy-sar-config" pod "25a92414-e34a-4711-b877-5decf9086277" (UID: "25a92414-e34a-4711-b877-5decf9086277"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:02:20.526357 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.526293 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25a92414-e34a-4711-b877-5decf9086277-cabundle-cert" pod "25a92414-e34a-4711-b877-5decf9086277" (UID: "25a92414-e34a-4711-b877-5decf9086277"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:02:20.528013 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.527983 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25a92414-e34a-4711-b877-5decf9086277-kube-api-access-z62v9" pod "25a92414-e34a-4711-b877-5decf9086277" (UID: "25a92414-e34a-4711-b877-5decf9086277"). InnerVolumeSpecName "kube-api-access-z62v9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:02:20.528068 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.527989 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25a92414-e34a-4711-b877-5decf9086277-proxy-tls" pod "25a92414-e34a-4711-b877-5decf9086277" (UID: "25a92414-e34a-4711-b877-5decf9086277"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:02:20.627446 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.627419 5054 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/25a92414-e34a-4711-b877-5decf9086277-proxy-tls\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:02:20.627446 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.627442 5054 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z62v9\" (UniqueName: \"kubernetes.io/projected/25a92414-e34a-4711-b877-5decf9086277-kube-api-access-z62v9\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:02:20.627603 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.627453 5054 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/25a92414-e34a-4711-b877-5decf9086277-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:02:20.627603 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.627462 5054 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/25a92414-e34a-4711-b877-5decf9086277-kserve-provision-location\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:02:20.627603 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.627471 5054 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/25a92414-e34a-4711-b877-5decf9086277-cabundle-cert\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:02:20.649404 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.649379 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-e43a086b9d_isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8_25a92414-e34a-4711-b877-5decf9086277/storage-initializer/1.log" Oct 30 19:02:20.649763 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.649748 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-gw1-e43a086b9d_isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8_25a92414-e34a-4711-b877-5decf9086277/storage-initializer/0.log" Oct 30 19:02:20.649815 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.649782 5054 generic.go:356] "Generic (PLEG): container finished" podID="25a92414-e34a-4711-b877-5decf9086277" containerID="42849f100d508b3b5dd5dff7d794266166d0184e8d243b54d546bfb24728d3f8" exitCode=1 Oct 30 19:02:20.649851 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.649828 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" event={"ID":"25a92414-e34a-4711-b877-5decf9086277","Type":"ContainerDied","Data":"42849f100d508b3b5dd5dff7d794266166d0184e8d243b54d546bfb24728d3f8"} Oct 30 19:02:20.649851 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.649841 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" Oct 30 19:02:20.649911 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.649881 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8" event={"ID":"25a92414-e34a-4711-b877-5decf9086277","Type":"ContainerDied","Data":"016f10071f85b2b455f39b6376a5441a0dd8f3e9beb2943b331a95a25eb17ccd"} Oct 30 19:02:20.649943 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.649911 5054 scope.go:122] "RemoveContainer" containerID="42849f100d508b3b5dd5dff7d794266166d0184e8d243b54d546bfb24728d3f8" Oct 30 19:02:20.658960 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.658943 5054 scope.go:122] "RemoveContainer" containerID="6a17f6b025e3a4158d6ffbdd51d1863f2789496e27de735af097ea5f662431fd" Oct 30 19:02:20.668476 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.668458 5054 scope.go:122] "RemoveContainer" containerID="42849f100d508b3b5dd5dff7d794266166d0184e8d243b54d546bfb24728d3f8" Oct 30 19:02:20.668912 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:02:20.668894 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42849f100d508b3b5dd5dff7d794266166d0184e8d243b54d546bfb24728d3f8\": container with ID starting with 42849f100d508b3b5dd5dff7d794266166d0184e8d243b54d546bfb24728d3f8 not found: ID does not exist" containerID="42849f100d508b3b5dd5dff7d794266166d0184e8d243b54d546bfb24728d3f8" Oct 30 19:02:20.668983 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.668924 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42849f100d508b3b5dd5dff7d794266166d0184e8d243b54d546bfb24728d3f8"} err="failed to get container status \"42849f100d508b3b5dd5dff7d794266166d0184e8d243b54d546bfb24728d3f8\": rpc error: code = NotFound desc = could not find container \"42849f100d508b3b5dd5dff7d794266166d0184e8d243b54d546bfb24728d3f8\": container with ID starting with 42849f100d508b3b5dd5dff7d794266166d0184e8d243b54d546bfb24728d3f8 not found: ID does not exist" Oct 30 19:02:20.668983 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.668949 5054 scope.go:122] "RemoveContainer" containerID="6a17f6b025e3a4158d6ffbdd51d1863f2789496e27de735af097ea5f662431fd" Oct 30 19:02:20.669248 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:02:20.669227 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a17f6b025e3a4158d6ffbdd51d1863f2789496e27de735af097ea5f662431fd\": container with ID starting with 6a17f6b025e3a4158d6ffbdd51d1863f2789496e27de735af097ea5f662431fd not found: ID does not exist" containerID="6a17f6b025e3a4158d6ffbdd51d1863f2789496e27de735af097ea5f662431fd" Oct 30 19:02:20.669291 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.669254 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a17f6b025e3a4158d6ffbdd51d1863f2789496e27de735af097ea5f662431fd"} err="failed to get container status \"6a17f6b025e3a4158d6ffbdd51d1863f2789496e27de735af097ea5f662431fd\": rpc error: code = NotFound desc = could not find container \"6a17f6b025e3a4158d6ffbdd51d1863f2789496e27de735af097ea5f662431fd\": container with ID starting with 6a17f6b025e3a4158d6ffbdd51d1863f2789496e27de735af097ea5f662431fd not found: ID does not exist" Oct 30 19:02:20.879602 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.879572 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8"] Oct 30 19:02:20.885520 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:20.885496 5054 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-init-fail-95186e-predictor-7bb44dcc5c-kk7s8"] Oct 30 19:02:22.420822 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:22.420792 5054 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="25a92414-e34a-4711-b877-5decf9086277" path="/var/lib/kubelet/pods/25a92414-e34a-4711-b877-5decf9086277/volumes" Oct 30 19:02:24.292486 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.292450 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt"] Oct 30 19:02:24.292967 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.292946 5054 cpu_manager.go:414] "containerMap: removing container" podUID="25a92414-e34a-4711-b877-5decf9086277" containerName="storage-initializer" Oct 30 19:02:24.293067 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.292970 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25a92414-e34a-4711-b877-5decf9086277" containerName="storage-initializer" Oct 30 19:02:24.293067 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.292992 5054 cpu_manager.go:414] "containerMap: removing container" podUID="25a92414-e34a-4711-b877-5decf9086277" containerName="storage-initializer" Oct 30 19:02:24.293067 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.293016 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25a92414-e34a-4711-b877-5decf9086277" containerName="storage-initializer" Oct 30 19:02:24.293244 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.293105 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="25a92414-e34a-4711-b877-5decf9086277" containerName="storage-initializer" Oct 30 19:02:24.293244 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.293117 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="25a92414-e34a-4711-b877-5decf9086277" containerName="storage-initializer" Oct 30 19:02:24.297822 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.297802 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:02:24.350508 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.350480 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"raw-sklearn-895a2-kube-rbac-proxy-sar-config\"" Oct 30 19:02:24.352886 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.352868 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw2rv\" (UniqueName: \"kubernetes.io/projected/e776e3a2-3a4a-4470-b352-59c3091b7ebb-kube-api-access-cw2rv\") pod \"raw-sklearn-895a2-predictor-6ffc444465-swqgt\" (UID: \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:02:24.352960 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.352907 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e776e3a2-3a4a-4470-b352-59c3091b7ebb-cabundle-cert\") pod \"raw-sklearn-895a2-predictor-6ffc444465-swqgt\" (UID: \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:02:24.352960 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.352932 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e776e3a2-3a4a-4470-b352-59c3091b7ebb-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-895a2-predictor-6ffc444465-swqgt\" (UID: \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:02:24.353077 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.353051 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e776e3a2-3a4a-4470-b352-59c3091b7ebb-proxy-tls\") pod \"raw-sklearn-895a2-predictor-6ffc444465-swqgt\" (UID: \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:02:24.353128 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.353095 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e776e3a2-3a4a-4470-b352-59c3091b7ebb-kserve-provision-location\") pod \"raw-sklearn-895a2-predictor-6ffc444465-swqgt\" (UID: \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:02:24.397324 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.397295 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"openshift-service-ca.crt\"" Oct 30 19:02:24.397324 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.397317 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"odh-kserve-custom-ca-bundle\"" Oct 30 19:02:24.397507 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.397457 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"kube-root-ca.crt\"" Oct 30 19:02:24.397612 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.397599 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"default-dockercfg-zc4kv\"" Oct 30 19:02:24.397722 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.397703 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"raw-sklearn-895a2-predictor-serving-cert\"" Oct 30 19:02:24.397774 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.397736 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"seaweedfs-s3-creds\"" Oct 30 19:02:24.399673 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.399651 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt"] Oct 30 19:02:24.453905 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.453879 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw2rv\" (UniqueName: \"kubernetes.io/projected/e776e3a2-3a4a-4470-b352-59c3091b7ebb-kube-api-access-cw2rv\") pod \"raw-sklearn-895a2-predictor-6ffc444465-swqgt\" (UID: \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:02:24.454041 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.453922 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e776e3a2-3a4a-4470-b352-59c3091b7ebb-cabundle-cert\") pod \"raw-sklearn-895a2-predictor-6ffc444465-swqgt\" (UID: \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:02:24.454041 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.453965 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e776e3a2-3a4a-4470-b352-59c3091b7ebb-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-895a2-predictor-6ffc444465-swqgt\" (UID: \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:02:24.454041 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.453992 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e776e3a2-3a4a-4470-b352-59c3091b7ebb-proxy-tls\") pod \"raw-sklearn-895a2-predictor-6ffc444465-swqgt\" (UID: \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:02:24.454172 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.454046 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e776e3a2-3a4a-4470-b352-59c3091b7ebb-kserve-provision-location\") pod \"raw-sklearn-895a2-predictor-6ffc444465-swqgt\" (UID: \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:02:24.454477 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.454447 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e776e3a2-3a4a-4470-b352-59c3091b7ebb-kserve-provision-location\") pod \"raw-sklearn-895a2-predictor-6ffc444465-swqgt\" (UID: \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:02:24.454676 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.454660 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e776e3a2-3a4a-4470-b352-59c3091b7ebb-cabundle-cert\") pod \"raw-sklearn-895a2-predictor-6ffc444465-swqgt\" (UID: \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:02:24.454725 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.454679 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e776e3a2-3a4a-4470-b352-59c3091b7ebb-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-895a2-predictor-6ffc444465-swqgt\" (UID: \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:02:24.456607 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.456579 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e776e3a2-3a4a-4470-b352-59c3091b7ebb-proxy-tls\") pod \"raw-sklearn-895a2-predictor-6ffc444465-swqgt\" (UID: \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:02:24.515306 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.515278 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw2rv\" (UniqueName: \"kubernetes.io/projected/e776e3a2-3a4a-4470-b352-59c3091b7ebb-kube-api-access-cw2rv\") pod \"raw-sklearn-895a2-predictor-6ffc444465-swqgt\" (UID: \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:02:24.614342 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:24.614301 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:02:25.044634 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:25.044549 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt"] Oct 30 19:02:25.050666 ip-10-0-132-74 kubenswrapper[5054]: W1030 19:02:25.050639 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode776e3a2_3a4a_4470_b352_59c3091b7ebb.slice/crio-783a2cadb3dcff4dad81d6ee3ce9a97c1b92cb7af2658a44dd84232fb118e8e8 WatchSource:0}: Error finding container 783a2cadb3dcff4dad81d6ee3ce9a97c1b92cb7af2658a44dd84232fb118e8e8: Status 404 returned error can't find the container with id 783a2cadb3dcff4dad81d6ee3ce9a97c1b92cb7af2658a44dd84232fb118e8e8 Oct 30 19:02:25.673687 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:25.673654 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" event={"ID":"e776e3a2-3a4a-4470-b352-59c3091b7ebb","Type":"ContainerStarted","Data":"ae61662823d8c298cac85200efa5a7b091010fbb1bd5694f68a16cce24e7ff86"} Oct 30 19:02:25.674105 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:25.673710 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" event={"ID":"e776e3a2-3a4a-4470-b352-59c3091b7ebb","Type":"ContainerStarted","Data":"783a2cadb3dcff4dad81d6ee3ce9a97c1b92cb7af2658a44dd84232fb118e8e8"} Oct 30 19:02:29.692849 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:29.692817 5054 generic.go:356] "Generic (PLEG): container finished" podID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerID="ae61662823d8c298cac85200efa5a7b091010fbb1bd5694f68a16cce24e7ff86" exitCode=0 Oct 30 19:02:29.693241 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:29.692884 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" event={"ID":"e776e3a2-3a4a-4470-b352-59c3091b7ebb","Type":"ContainerDied","Data":"ae61662823d8c298cac85200efa5a7b091010fbb1bd5694f68a16cce24e7ff86"} Oct 30 19:02:30.699359 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:30.699323 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" event={"ID":"e776e3a2-3a4a-4470-b352-59c3091b7ebb","Type":"ContainerStarted","Data":"6001f2b8872d7cbccea79adf8d62d756884ed8d619036892722591dd2cd9dcc9"} Oct 30 19:02:30.699858 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:30.699397 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" event={"ID":"e776e3a2-3a4a-4470-b352-59c3091b7ebb","Type":"ContainerStarted","Data":"0dbbcc3e832a93048cdec62c0baa5903052d3ed28639807130f8e79d2effa50e"} Oct 30 19:02:30.699858 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:30.699607 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:02:30.821456 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:30.821410 5054 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" podStartSLOduration=3.086081411 podStartE2EDuration="6.821397708s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.735316297s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:30.820871027 +0000 UTC m=+1492.549249745" watchObservedRunningTime="2026-10-30 19:02:30.821397708 +0000 UTC m=+1492.549776423" Oct 30 19:02:31.709427 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:31.709395 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:02:31.710422 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:31.710397 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" podUID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused" Oct 30 19:02:32.713655 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:32.713602 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" podUID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused" Oct 30 19:02:37.717954 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:37.717923 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:02:37.718427 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:37.718375 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" podUID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused" Oct 30 19:02:38.547342 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:38.547215 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 19:02:38.564201 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:38.553233 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 19:02:38.598196 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:02:38.598164 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:02:47.719110 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:47.719068 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" podUID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused" Oct 30 19:02:57.718336 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:02:57.718298 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" podUID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused" Oct 30 19:03:07.718507 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:07.718464 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" podUID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused" Oct 30 19:03:08.600128 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:03:08.600090 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:03:17.718421 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:17.718376 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" podUID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused" Oct 30 19:03:27.719300 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:27.719264 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" podUID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused" Oct 30 19:03:37.718509 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:37.718465 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" podUID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused" Oct 30 19:03:38.597294 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:03:38.597257 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:03:47.718920 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:47.718883 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:03:54.825521 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:54.825490 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt"] Oct 30 19:03:54.825949 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:54.825734 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" podUID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerName="kserve-container" containerID="cri-o://0dbbcc3e832a93048cdec62c0baa5903052d3ed28639807130f8e79d2effa50e" gracePeriod=30 Oct 30 19:03:54.825949 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:54.825750 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" podUID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerName="kube-rbac-proxy" containerID="cri-o://6001f2b8872d7cbccea79adf8d62d756884ed8d619036892722591dd2cd9dcc9" gracePeriod=30 Oct 30 19:03:55.037972 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:55.037902 5054 generic.go:356] "Generic (PLEG): container finished" podID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerID="6001f2b8872d7cbccea79adf8d62d756884ed8d619036892722591dd2cd9dcc9" exitCode=0 Oct 30 19:03:55.038117 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:55.037973 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" event={"ID":"e776e3a2-3a4a-4470-b352-59c3091b7ebb","Type":"ContainerDied","Data":"6001f2b8872d7cbccea79adf8d62d756884ed8d619036892722591dd2cd9dcc9"} Oct 30 19:03:57.713837 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:57.713795 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" podUID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.44:8643/healthz\": dial tcp 10.132.0.44:8643: connect: connection refused" Oct 30 19:03:57.719139 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:57.719114 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" podUID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.44:8080: connect: connection refused" Oct 30 19:03:59.468242 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:59.468217 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:03:59.522764 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:59.522704 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e776e3a2-3a4a-4470-b352-59c3091b7ebb-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e776e3a2-3a4a-4470-b352-59c3091b7ebb-kube-rbac-proxy-sar-config\") pod \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\" (UID: \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\") " Oct 30 19:03:59.522764 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:59.522756 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e776e3a2-3a4a-4470-b352-59c3091b7ebb-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e776e3a2-3a4a-4470-b352-59c3091b7ebb-cabundle-cert\") pod \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\" (UID: \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\") " Oct 30 19:03:59.522952 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:59.522814 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e776e3a2-3a4a-4470-b352-59c3091b7ebb-proxy-tls\" (UniqueName: \"kubernetes.io/secret/e776e3a2-3a4a-4470-b352-59c3091b7ebb-proxy-tls\") pod \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\" (UID: \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\") " Oct 30 19:03:59.522952 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:59.522845 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e776e3a2-3a4a-4470-b352-59c3091b7ebb-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e776e3a2-3a4a-4470-b352-59c3091b7ebb-kserve-provision-location\") pod \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\" (UID: \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\") " Oct 30 19:03:59.522952 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:59.522902 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e776e3a2-3a4a-4470-b352-59c3091b7ebb-kube-api-access-cw2rv\" (UniqueName: \"kubernetes.io/projected/e776e3a2-3a4a-4470-b352-59c3091b7ebb-kube-api-access-cw2rv\") pod \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\" (UID: \"e776e3a2-3a4a-4470-b352-59c3091b7ebb\") " Oct 30 19:03:59.523173 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:59.523128 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e776e3a2-3a4a-4470-b352-59c3091b7ebb-kube-rbac-proxy-sar-config" pod "e776e3a2-3a4a-4470-b352-59c3091b7ebb" (UID: "e776e3a2-3a4a-4470-b352-59c3091b7ebb"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:03:59.523173 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:59.523147 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e776e3a2-3a4a-4470-b352-59c3091b7ebb-cabundle-cert" pod "e776e3a2-3a4a-4470-b352-59c3091b7ebb" (UID: "e776e3a2-3a4a-4470-b352-59c3091b7ebb"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:03:59.523351 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:59.523203 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e776e3a2-3a4a-4470-b352-59c3091b7ebb-kserve-provision-location" pod "e776e3a2-3a4a-4470-b352-59c3091b7ebb" (UID: "e776e3a2-3a4a-4470-b352-59c3091b7ebb"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:03:59.523351 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:59.523228 5054 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e776e3a2-3a4a-4470-b352-59c3091b7ebb-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:03:59.523351 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:59.523249 5054 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e776e3a2-3a4a-4470-b352-59c3091b7ebb-cabundle-cert\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:03:59.524834 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:59.524810 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e776e3a2-3a4a-4470-b352-59c3091b7ebb-proxy-tls" pod "e776e3a2-3a4a-4470-b352-59c3091b7ebb" (UID: "e776e3a2-3a4a-4470-b352-59c3091b7ebb"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:03:59.524965 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:59.524941 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e776e3a2-3a4a-4470-b352-59c3091b7ebb-kube-api-access-cw2rv" pod "e776e3a2-3a4a-4470-b352-59c3091b7ebb" (UID: "e776e3a2-3a4a-4470-b352-59c3091b7ebb"). InnerVolumeSpecName "kube-api-access-cw2rv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:03:59.624185 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:59.624159 5054 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e776e3a2-3a4a-4470-b352-59c3091b7ebb-proxy-tls\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:03:59.624185 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:59.624181 5054 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e776e3a2-3a4a-4470-b352-59c3091b7ebb-kserve-provision-location\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:03:59.624185 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:03:59.624191 5054 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cw2rv\" (UniqueName: \"kubernetes.io/projected/e776e3a2-3a4a-4470-b352-59c3091b7ebb-kube-api-access-cw2rv\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:04:00.060772 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:00.060734 5054 generic.go:356] "Generic (PLEG): container finished" podID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerID="0dbbcc3e832a93048cdec62c0baa5903052d3ed28639807130f8e79d2effa50e" exitCode=0 Oct 30 19:04:00.060976 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:00.060798 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" Oct 30 19:04:00.060976 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:00.060821 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" event={"ID":"e776e3a2-3a4a-4470-b352-59c3091b7ebb","Type":"ContainerDied","Data":"0dbbcc3e832a93048cdec62c0baa5903052d3ed28639807130f8e79d2effa50e"} Oct 30 19:04:00.060976 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:00.060878 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt" event={"ID":"e776e3a2-3a4a-4470-b352-59c3091b7ebb","Type":"ContainerDied","Data":"783a2cadb3dcff4dad81d6ee3ce9a97c1b92cb7af2658a44dd84232fb118e8e8"} Oct 30 19:04:00.060976 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:00.060903 5054 scope.go:122] "RemoveContainer" containerID="6001f2b8872d7cbccea79adf8d62d756884ed8d619036892722591dd2cd9dcc9" Oct 30 19:04:00.069349 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:00.069332 5054 scope.go:122] "RemoveContainer" containerID="0dbbcc3e832a93048cdec62c0baa5903052d3ed28639807130f8e79d2effa50e" Oct 30 19:04:00.076471 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:00.076451 5054 scope.go:122] "RemoveContainer" containerID="ae61662823d8c298cac85200efa5a7b091010fbb1bd5694f68a16cce24e7ff86" Oct 30 19:04:00.083413 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:00.083395 5054 scope.go:122] "RemoveContainer" containerID="6001f2b8872d7cbccea79adf8d62d756884ed8d619036892722591dd2cd9dcc9" Oct 30 19:04:00.083661 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:04:00.083641 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6001f2b8872d7cbccea79adf8d62d756884ed8d619036892722591dd2cd9dcc9\": container with ID starting with 6001f2b8872d7cbccea79adf8d62d756884ed8d619036892722591dd2cd9dcc9 not found: ID does not exist" containerID="6001f2b8872d7cbccea79adf8d62d756884ed8d619036892722591dd2cd9dcc9" Oct 30 19:04:00.083706 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:00.083671 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6001f2b8872d7cbccea79adf8d62d756884ed8d619036892722591dd2cd9dcc9"} err="failed to get container status \"6001f2b8872d7cbccea79adf8d62d756884ed8d619036892722591dd2cd9dcc9\": rpc error: code = NotFound desc = could not find container \"6001f2b8872d7cbccea79adf8d62d756884ed8d619036892722591dd2cd9dcc9\": container with ID starting with 6001f2b8872d7cbccea79adf8d62d756884ed8d619036892722591dd2cd9dcc9 not found: ID does not exist" Oct 30 19:04:00.083706 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:00.083690 5054 scope.go:122] "RemoveContainer" containerID="0dbbcc3e832a93048cdec62c0baa5903052d3ed28639807130f8e79d2effa50e" Oct 30 19:04:00.083937 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:04:00.083914 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0dbbcc3e832a93048cdec62c0baa5903052d3ed28639807130f8e79d2effa50e\": container with ID starting with 0dbbcc3e832a93048cdec62c0baa5903052d3ed28639807130f8e79d2effa50e not found: ID does not exist" containerID="0dbbcc3e832a93048cdec62c0baa5903052d3ed28639807130f8e79d2effa50e" Oct 30 19:04:00.084044 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:00.083941 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0dbbcc3e832a93048cdec62c0baa5903052d3ed28639807130f8e79d2effa50e"} err="failed to get container status \"0dbbcc3e832a93048cdec62c0baa5903052d3ed28639807130f8e79d2effa50e\": rpc error: code = NotFound desc = could not find container \"0dbbcc3e832a93048cdec62c0baa5903052d3ed28639807130f8e79d2effa50e\": container with ID starting with 0dbbcc3e832a93048cdec62c0baa5903052d3ed28639807130f8e79d2effa50e not found: ID does not exist" Oct 30 19:04:00.084044 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:00.083956 5054 scope.go:122] "RemoveContainer" containerID="ae61662823d8c298cac85200efa5a7b091010fbb1bd5694f68a16cce24e7ff86" Oct 30 19:04:00.084190 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:04:00.084175 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae61662823d8c298cac85200efa5a7b091010fbb1bd5694f68a16cce24e7ff86\": container with ID starting with ae61662823d8c298cac85200efa5a7b091010fbb1bd5694f68a16cce24e7ff86 not found: ID does not exist" containerID="ae61662823d8c298cac85200efa5a7b091010fbb1bd5694f68a16cce24e7ff86" Oct 30 19:04:00.084225 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:00.084193 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae61662823d8c298cac85200efa5a7b091010fbb1bd5694f68a16cce24e7ff86"} err="failed to get container status \"ae61662823d8c298cac85200efa5a7b091010fbb1bd5694f68a16cce24e7ff86\": rpc error: code = NotFound desc = could not find container \"ae61662823d8c298cac85200efa5a7b091010fbb1bd5694f68a16cce24e7ff86\": container with ID starting with ae61662823d8c298cac85200efa5a7b091010fbb1bd5694f68a16cce24e7ff86 not found: ID does not exist" Oct 30 19:04:00.222654 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:00.222629 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt"] Oct 30 19:04:00.271143 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:00.271118 5054 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-e43a086b9d/raw-sklearn-895a2-predictor-6ffc444465-swqgt"] Oct 30 19:04:00.422278 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:00.422245 5054 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" path="/var/lib/kubelet/pods/e776e3a2-3a4a-4470-b352-59c3091b7ebb/volumes" Oct 30 19:04:03.958681 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:03.958651 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx"] Oct 30 19:04:03.959091 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:03.958942 5054 cpu_manager.go:414] "containerMap: removing container" podUID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerName="storage-initializer" Oct 30 19:04:03.959091 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:03.958952 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerName="storage-initializer" Oct 30 19:04:03.959091 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:03.958963 5054 cpu_manager.go:414] "containerMap: removing container" podUID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerName="kserve-container" Oct 30 19:04:03.959091 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:03.958968 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerName="kserve-container" Oct 30 19:04:03.959091 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:03.958985 5054 cpu_manager.go:414] "containerMap: removing container" podUID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerName="kube-rbac-proxy" Oct 30 19:04:03.959091 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:03.958990 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerName="kube-rbac-proxy" Oct 30 19:04:03.959091 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:03.959066 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerName="kserve-container" Oct 30 19:04:03.959091 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:03.959077 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="e776e3a2-3a4a-4470-b352-59c3091b7ebb" containerName="kube-rbac-proxy" Oct 30 19:04:03.963712 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:03.963693 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:04:04.056251 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.056221 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx\" (UID: \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:04:04.056395 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.056277 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-cabundle-cert\") pod \"raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx\" (UID: \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:04:04.056395 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.056349 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-proxy-tls\") pod \"raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx\" (UID: \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:04:04.056395 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.056382 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-424rw\" (UniqueName: \"kubernetes.io/projected/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-kube-api-access-424rw\") pod \"raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx\" (UID: \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:04:04.056520 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.056426 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-kserve-provision-location\") pod \"raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx\" (UID: \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:04:04.062965 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.062942 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"raw-sklearn-runtime-67a06-kube-rbac-proxy-sar-config\"" Oct 30 19:04:04.063083 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.062983 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"seaweedfs-s3-creds\"" Oct 30 19:04:04.063083 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.062998 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"kube-root-ca.crt\"" Oct 30 19:04:04.063083 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.063024 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"odh-kserve-custom-ca-bundle\"" Oct 30 19:04:04.063083 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.063036 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"openshift-service-ca.crt\"" Oct 30 19:04:04.063083 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.063042 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"raw-sklearn-runtime-67a06-predictor-serving-cert\"" Oct 30 19:04:04.063268 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.063141 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"default-dockercfg-zc4kv\"" Oct 30 19:04:04.065370 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.065352 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx"] Oct 30 19:04:04.156951 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.156922 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-cabundle-cert\") pod \"raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx\" (UID: \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:04:04.157118 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.156955 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-proxy-tls\") pod \"raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx\" (UID: \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:04:04.157118 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.156984 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-424rw\" (UniqueName: \"kubernetes.io/projected/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-kube-api-access-424rw\") pod \"raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx\" (UID: \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:04:04.157118 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.157029 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-kserve-provision-location\") pod \"raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx\" (UID: \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:04:04.157118 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.157071 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx\" (UID: \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:04:04.157549 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.157527 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-kserve-provision-location\") pod \"raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx\" (UID: \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:04:04.157642 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.157587 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-cabundle-cert\") pod \"raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx\" (UID: \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:04:04.157726 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.157705 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx\" (UID: \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:04:04.159447 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.159425 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-proxy-tls\") pod \"raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx\" (UID: \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:04:04.218044 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.217963 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-424rw\" (UniqueName: \"kubernetes.io/projected/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-kube-api-access-424rw\") pod \"raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx\" (UID: \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\") " pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:04:04.280499 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.280473 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:04:04.700489 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:04.700459 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx"] Oct 30 19:04:04.706704 ip-10-0-132-74 kubenswrapper[5054]: W1030 19:04:04.706667 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b0983a4_50e6_4296_b8aa_0243dd9e1e0a.slice/crio-312ae6d5f9d56284e55b5f9227398c3ed37cb7935b1a4ccb4e2012c813d265a3 WatchSource:0}: Error finding container 312ae6d5f9d56284e55b5f9227398c3ed37cb7935b1a4ccb4e2012c813d265a3: Status 404 returned error can't find the container with id 312ae6d5f9d56284e55b5f9227398c3ed37cb7935b1a4ccb4e2012c813d265a3 Oct 30 19:04:05.083793 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:05.083714 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" event={"ID":"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a","Type":"ContainerStarted","Data":"55db8c8320121dc08e721f0911bec504c3794ccc5b657092e501d75b6cc5a69a"} Oct 30 19:04:05.083793 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:05.083771 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" event={"ID":"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a","Type":"ContainerStarted","Data":"312ae6d5f9d56284e55b5f9227398c3ed37cb7935b1a4ccb4e2012c813d265a3"} Oct 30 19:04:08.597024 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:04:08.596978 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:04:09.100396 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:09.100358 5054 generic.go:356] "Generic (PLEG): container finished" podID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerID="55db8c8320121dc08e721f0911bec504c3794ccc5b657092e501d75b6cc5a69a" exitCode=0 Oct 30 19:04:09.100564 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:09.100409 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" event={"ID":"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a","Type":"ContainerDied","Data":"55db8c8320121dc08e721f0911bec504c3794ccc5b657092e501d75b6cc5a69a"} Oct 30 19:04:10.107272 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:10.107238 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" event={"ID":"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a","Type":"ContainerStarted","Data":"e83c90871849a2391540020d9bb6aa06de896b9033e2eec1da5d85d24a8a3b11"} Oct 30 19:04:10.107656 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:10.107297 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" event={"ID":"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a","Type":"ContainerStarted","Data":"c3d3906df5812e68e9d468bde2f570a58d832f46842d656501e7fd204a8da34b"} Oct 30 19:04:10.107656 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:10.107533 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:04:10.228599 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:10.228545 5054 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" podStartSLOduration=3.239648696 podStartE2EDuration="7.228530544s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.988881848s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:04:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:04:10.227112061 +0000 UTC m=+1591.955490768" watchObservedRunningTime="2026-10-30 19:04:10.228530544 +0000 UTC m=+1591.956909334" Oct 30 19:04:11.112815 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:11.112785 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:04:11.114069 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:11.114036 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" podUID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.45:8080: connect: connection refused" Oct 30 19:04:12.117345 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:12.117304 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" podUID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.45:8080: connect: connection refused" Oct 30 19:04:17.121720 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:17.121692 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:04:17.122209 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:17.122182 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" podUID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.45:8080: connect: connection refused" Oct 30 19:04:27.122432 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:27.122393 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" podUID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.45:8080: connect: connection refused" Oct 30 19:04:37.122877 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:37.122841 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" podUID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.45:8080: connect: connection refused" Oct 30 19:04:38.597338 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:04:38.597301 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:04:47.122303 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:47.122256 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" podUID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.45:8080: connect: connection refused" Oct 30 19:04:57.122849 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:04:57.122807 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" podUID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.45:8080: connect: connection refused" Oct 30 19:05:07.122160 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:07.122123 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" podUID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.45:8080: connect: connection refused" Oct 30 19:05:08.597701 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:05:08.597656 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:05:17.123032 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:17.122980 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" podUID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.45:8080: connect: connection refused" Oct 30 19:05:19.416433 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:19.416342 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" podUID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.45:8080: connect: connection refused" Oct 30 19:05:29.417069 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:29.417025 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:05:34.302253 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:34.302052 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx"] Oct 30 19:05:34.302753 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:34.302446 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" podUID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerName="kserve-container" containerID="cri-o://c3d3906df5812e68e9d468bde2f570a58d832f46842d656501e7fd204a8da34b" gracePeriod=30 Oct 30 19:05:34.302753 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:34.302685 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" podUID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerName="kube-rbac-proxy" containerID="cri-o://e83c90871849a2391540020d9bb6aa06de896b9033e2eec1da5d85d24a8a3b11" gracePeriod=30 Oct 30 19:05:34.445466 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:34.445436 5054 generic.go:356] "Generic (PLEG): container finished" podID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerID="e83c90871849a2391540020d9bb6aa06de896b9033e2eec1da5d85d24a8a3b11" exitCode=0 Oct 30 19:05:34.445585 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:34.445503 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" event={"ID":"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a","Type":"ContainerDied","Data":"e83c90871849a2391540020d9bb6aa06de896b9033e2eec1da5d85d24a8a3b11"} Oct 30 19:05:37.117879 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:37.117837 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" podUID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.45:8643/healthz\": dial tcp 10.132.0.45:8643: connect: connection refused" Oct 30 19:05:38.596787 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:05:38.596754 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:05:38.949481 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:38.949457 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:05:39.117488 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.117463 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-kserve-provision-location\") pod \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\" (UID: \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\") " Oct 30 19:05:39.117641 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.117511 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-proxy-tls\" (UniqueName: \"kubernetes.io/secret/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-proxy-tls\") pod \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\" (UID: \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\") " Oct 30 19:05:39.117641 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.117585 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-kube-api-access-424rw\" (UniqueName: \"kubernetes.io/projected/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-kube-api-access-424rw\") pod \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\" (UID: \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\") " Oct 30 19:05:39.117641 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.117614 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-kube-rbac-proxy-sar-config\") pod \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\" (UID: \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\") " Oct 30 19:05:39.117763 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.117659 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-cabundle-cert\") pod \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\" (UID: \"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a\") " Oct 30 19:05:39.117953 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.117855 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-kserve-provision-location" pod "8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" (UID: "8b0983a4-50e6-4296-b8aa-0243dd9e1e0a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:05:39.118235 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.118067 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-kube-rbac-proxy-sar-config" pod "8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" (UID: "8b0983a4-50e6-4296-b8aa-0243dd9e1e0a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:05:39.118235 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.118077 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-cabundle-cert" pod "8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" (UID: "8b0983a4-50e6-4296-b8aa-0243dd9e1e0a"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:05:39.119747 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.119701 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-kube-api-access-424rw" pod "8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" (UID: "8b0983a4-50e6-4296-b8aa-0243dd9e1e0a"). InnerVolumeSpecName "kube-api-access-424rw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:05:39.120175 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.120158 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-proxy-tls" pod "8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" (UID: "8b0983a4-50e6-4296-b8aa-0243dd9e1e0a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:05:39.218892 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.218867 5054 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-kserve-provision-location\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:05:39.218892 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.218889 5054 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-proxy-tls\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:05:39.219061 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.218900 5054 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-424rw\" (UniqueName: \"kubernetes.io/projected/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-kube-api-access-424rw\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:05:39.219061 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.218909 5054 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:05:39.219061 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.218918 5054 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a-cabundle-cert\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:05:39.467730 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.467643 5054 generic.go:356] "Generic (PLEG): container finished" podID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerID="c3d3906df5812e68e9d468bde2f570a58d832f46842d656501e7fd204a8da34b" exitCode=0 Oct 30 19:05:39.467730 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.467702 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" Oct 30 19:05:39.467730 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.467711 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" event={"ID":"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a","Type":"ContainerDied","Data":"c3d3906df5812e68e9d468bde2f570a58d832f46842d656501e7fd204a8da34b"} Oct 30 19:05:39.468025 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.467757 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx" event={"ID":"8b0983a4-50e6-4296-b8aa-0243dd9e1e0a","Type":"ContainerDied","Data":"312ae6d5f9d56284e55b5f9227398c3ed37cb7935b1a4ccb4e2012c813d265a3"} Oct 30 19:05:39.468025 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.467782 5054 scope.go:122] "RemoveContainer" containerID="e83c90871849a2391540020d9bb6aa06de896b9033e2eec1da5d85d24a8a3b11" Oct 30 19:05:39.475911 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.475893 5054 scope.go:122] "RemoveContainer" containerID="c3d3906df5812e68e9d468bde2f570a58d832f46842d656501e7fd204a8da34b" Oct 30 19:05:39.483059 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.483043 5054 scope.go:122] "RemoveContainer" containerID="55db8c8320121dc08e721f0911bec504c3794ccc5b657092e501d75b6cc5a69a" Oct 30 19:05:39.490117 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.490093 5054 scope.go:122] "RemoveContainer" containerID="e83c90871849a2391540020d9bb6aa06de896b9033e2eec1da5d85d24a8a3b11" Oct 30 19:05:39.490377 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:05:39.490356 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e83c90871849a2391540020d9bb6aa06de896b9033e2eec1da5d85d24a8a3b11\": container with ID starting with e83c90871849a2391540020d9bb6aa06de896b9033e2eec1da5d85d24a8a3b11 not found: ID does not exist" containerID="e83c90871849a2391540020d9bb6aa06de896b9033e2eec1da5d85d24a8a3b11" Oct 30 19:05:39.490440 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.490385 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e83c90871849a2391540020d9bb6aa06de896b9033e2eec1da5d85d24a8a3b11"} err="failed to get container status \"e83c90871849a2391540020d9bb6aa06de896b9033e2eec1da5d85d24a8a3b11\": rpc error: code = NotFound desc = could not find container \"e83c90871849a2391540020d9bb6aa06de896b9033e2eec1da5d85d24a8a3b11\": container with ID starting with e83c90871849a2391540020d9bb6aa06de896b9033e2eec1da5d85d24a8a3b11 not found: ID does not exist" Oct 30 19:05:39.490440 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.490403 5054 scope.go:122] "RemoveContainer" containerID="c3d3906df5812e68e9d468bde2f570a58d832f46842d656501e7fd204a8da34b" Oct 30 19:05:39.490641 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:05:39.490623 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3d3906df5812e68e9d468bde2f570a58d832f46842d656501e7fd204a8da34b\": container with ID starting with c3d3906df5812e68e9d468bde2f570a58d832f46842d656501e7fd204a8da34b not found: ID does not exist" containerID="c3d3906df5812e68e9d468bde2f570a58d832f46842d656501e7fd204a8da34b" Oct 30 19:05:39.490702 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.490650 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3d3906df5812e68e9d468bde2f570a58d832f46842d656501e7fd204a8da34b"} err="failed to get container status \"c3d3906df5812e68e9d468bde2f570a58d832f46842d656501e7fd204a8da34b\": rpc error: code = NotFound desc = could not find container \"c3d3906df5812e68e9d468bde2f570a58d832f46842d656501e7fd204a8da34b\": container with ID starting with c3d3906df5812e68e9d468bde2f570a58d832f46842d656501e7fd204a8da34b not found: ID does not exist" Oct 30 19:05:39.490702 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.490672 5054 scope.go:122] "RemoveContainer" containerID="55db8c8320121dc08e721f0911bec504c3794ccc5b657092e501d75b6cc5a69a" Oct 30 19:05:39.490882 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:05:39.490868 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55db8c8320121dc08e721f0911bec504c3794ccc5b657092e501d75b6cc5a69a\": container with ID starting with 55db8c8320121dc08e721f0911bec504c3794ccc5b657092e501d75b6cc5a69a not found: ID does not exist" containerID="55db8c8320121dc08e721f0911bec504c3794ccc5b657092e501d75b6cc5a69a" Oct 30 19:05:39.490919 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.490885 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55db8c8320121dc08e721f0911bec504c3794ccc5b657092e501d75b6cc5a69a"} err="failed to get container status \"55db8c8320121dc08e721f0911bec504c3794ccc5b657092e501d75b6cc5a69a\": rpc error: code = NotFound desc = could not find container \"55db8c8320121dc08e721f0911bec504c3794ccc5b657092e501d75b6cc5a69a\": container with ID starting with 55db8c8320121dc08e721f0911bec504c3794ccc5b657092e501d75b6cc5a69a not found: ID does not exist" Oct 30 19:05:39.630867 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.630841 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx"] Oct 30 19:05:39.678708 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:39.678682 5054 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-e43a086b9d/raw-sklearn-runtime-67a06-predictor-6d744fbb5c-845tx"] Oct 30 19:05:40.421200 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:40.421167 5054 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" path="/var/lib/kubelet/pods/8b0983a4-50e6-4296-b8aa-0243dd9e1e0a/volumes" Oct 30 19:05:43.396718 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.396687 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r"] Oct 30 19:05:43.397087 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.396986 5054 cpu_manager.go:414] "containerMap: removing container" podUID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerName="kserve-container" Oct 30 19:05:43.397087 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.396996 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerName="kserve-container" Oct 30 19:05:43.397087 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.397021 5054 cpu_manager.go:414] "containerMap: removing container" podUID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerName="kube-rbac-proxy" Oct 30 19:05:43.397087 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.397027 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerName="kube-rbac-proxy" Oct 30 19:05:43.397087 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.397044 5054 cpu_manager.go:414] "containerMap: removing container" podUID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerName="storage-initializer" Oct 30 19:05:43.397087 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.397049 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerName="storage-initializer" Oct 30 19:05:43.397278 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.397113 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerName="kserve-container" Oct 30 19:05:43.397278 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.397120 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b0983a4-50e6-4296-b8aa-0243dd9e1e0a" containerName="kube-rbac-proxy" Oct 30 19:05:43.401822 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.401799 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:05:43.454464 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.454437 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"kube-root-ca.crt\"" Oct 30 19:05:43.455581 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.455560 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1ac051df-4fd9-441c-af3e-06e6517ee926-kserve-provision-location\") pod \"isvc-canary-create-v2-predictor-587db58d45-jtq2r\" (UID: \"1ac051df-4fd9-441c-af3e-06e6517ee926\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:05:43.455641 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.455617 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1ac051df-4fd9-441c-af3e-06e6517ee926-proxy-tls\") pod \"isvc-canary-create-v2-predictor-587db58d45-jtq2r\" (UID: \"1ac051df-4fd9-441c-af3e-06e6517ee926\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:05:43.455740 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.455722 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1ac051df-4fd9-441c-af3e-06e6517ee926-cabundle-cert\") pod \"isvc-canary-create-v2-predictor-587db58d45-jtq2r\" (UID: \"1ac051df-4fd9-441c-af3e-06e6517ee926\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:05:43.455776 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.455755 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pckl\" (UniqueName: \"kubernetes.io/projected/1ac051df-4fd9-441c-af3e-06e6517ee926-kube-api-access-5pckl\") pod \"isvc-canary-create-v2-predictor-587db58d45-jtq2r\" (UID: \"1ac051df-4fd9-441c-af3e-06e6517ee926\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:05:43.455841 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.455828 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1ac051df-4fd9-441c-af3e-06e6517ee926-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-v2-predictor-587db58d45-jtq2r\" (UID: \"1ac051df-4fd9-441c-af3e-06e6517ee926\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:05:43.500651 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.500615 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"default-dockercfg-zc4kv\"" Oct 30 19:05:43.500795 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.500665 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"isvc-canary-create-kube-rbac-proxy-sar-config\"" Oct 30 19:05:43.500795 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.500671 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"odh-kserve-custom-ca-bundle\"" Oct 30 19:05:43.500795 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.500671 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"isvc-canary-create-v2-predictor-serving-cert\"" Oct 30 19:05:43.500795 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.500714 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"seaweedfs-s3-creds\"" Oct 30 19:05:43.500795 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.500722 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"openshift-service-ca.crt\"" Oct 30 19:05:43.502133 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.502117 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r"] Oct 30 19:05:43.556802 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.556781 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1ac051df-4fd9-441c-af3e-06e6517ee926-proxy-tls\") pod \"isvc-canary-create-v2-predictor-587db58d45-jtq2r\" (UID: \"1ac051df-4fd9-441c-af3e-06e6517ee926\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:05:43.556946 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.556811 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1ac051df-4fd9-441c-af3e-06e6517ee926-cabundle-cert\") pod \"isvc-canary-create-v2-predictor-587db58d45-jtq2r\" (UID: \"1ac051df-4fd9-441c-af3e-06e6517ee926\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:05:43.556946 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.556831 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pckl\" (UniqueName: \"kubernetes.io/projected/1ac051df-4fd9-441c-af3e-06e6517ee926-kube-api-access-5pckl\") pod \"isvc-canary-create-v2-predictor-587db58d45-jtq2r\" (UID: \"1ac051df-4fd9-441c-af3e-06e6517ee926\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:05:43.556946 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.556851 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1ac051df-4fd9-441c-af3e-06e6517ee926-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-v2-predictor-587db58d45-jtq2r\" (UID: \"1ac051df-4fd9-441c-af3e-06e6517ee926\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:05:43.557176 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.557040 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1ac051df-4fd9-441c-af3e-06e6517ee926-kserve-provision-location\") pod \"isvc-canary-create-v2-predictor-587db58d45-jtq2r\" (UID: \"1ac051df-4fd9-441c-af3e-06e6517ee926\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:05:43.557411 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.557385 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1ac051df-4fd9-441c-af3e-06e6517ee926-kserve-provision-location\") pod \"isvc-canary-create-v2-predictor-587db58d45-jtq2r\" (UID: \"1ac051df-4fd9-441c-af3e-06e6517ee926\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:05:43.557710 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.557693 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1ac051df-4fd9-441c-af3e-06e6517ee926-cabundle-cert\") pod \"isvc-canary-create-v2-predictor-587db58d45-jtq2r\" (UID: \"1ac051df-4fd9-441c-af3e-06e6517ee926\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:05:43.557779 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.557763 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1ac051df-4fd9-441c-af3e-06e6517ee926-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-v2-predictor-587db58d45-jtq2r\" (UID: \"1ac051df-4fd9-441c-af3e-06e6517ee926\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:05:43.559301 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.559281 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1ac051df-4fd9-441c-af3e-06e6517ee926-proxy-tls\") pod \"isvc-canary-create-v2-predictor-587db58d45-jtq2r\" (UID: \"1ac051df-4fd9-441c-af3e-06e6517ee926\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:05:43.617925 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.617897 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pckl\" (UniqueName: \"kubernetes.io/projected/1ac051df-4fd9-441c-af3e-06e6517ee926-kube-api-access-5pckl\") pod \"isvc-canary-create-v2-predictor-587db58d45-jtq2r\" (UID: \"1ac051df-4fd9-441c-af3e-06e6517ee926\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:05:43.719608 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:43.719551 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:05:44.140147 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.140113 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r"] Oct 30 19:05:44.145880 ip-10-0-132-74 kubenswrapper[5054]: W1030 19:05:44.145859 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ac051df_4fd9_441c_af3e_06e6517ee926.slice/crio-f38704884dd4b8bd41e8a0dba71750ac89320b36906a30f1a36f246b63d4b491 WatchSource:0}: Error finding container f38704884dd4b8bd41e8a0dba71750ac89320b36906a30f1a36f246b63d4b491: Status 404 returned error can't find the container with id f38704884dd4b8bd41e8a0dba71750ac89320b36906a30f1a36f246b63d4b491 Oct 30 19:05:44.357432 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.357397 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5"] Oct 30 19:05:44.360920 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.360903 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:05:44.413937 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.413878 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"isvc-canary-create-predictor-serving-cert\"" Oct 30 19:05:44.423832 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.423807 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5"] Oct 30 19:05:44.462981 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.462952 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tnck\" (UniqueName: \"kubernetes.io/projected/1c909efe-4bc0-46d8-9246-d12a5250d5b6-kube-api-access-9tnck\") pod \"isvc-canary-create-predictor-8b6998fb4-b8wv5\" (UID: \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:05:44.463153 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.463020 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1c909efe-4bc0-46d8-9246-d12a5250d5b6-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-8b6998fb4-b8wv5\" (UID: \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:05:44.463153 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.463092 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1c909efe-4bc0-46d8-9246-d12a5250d5b6-kserve-provision-location\") pod \"isvc-canary-create-predictor-8b6998fb4-b8wv5\" (UID: \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:05:44.463284 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.463177 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1c909efe-4bc0-46d8-9246-d12a5250d5b6-proxy-tls\") pod \"isvc-canary-create-predictor-8b6998fb4-b8wv5\" (UID: \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:05:44.463284 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.463238 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1c909efe-4bc0-46d8-9246-d12a5250d5b6-cabundle-cert\") pod \"isvc-canary-create-predictor-8b6998fb4-b8wv5\" (UID: \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:05:44.490624 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.490594 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" event={"ID":"1ac051df-4fd9-441c-af3e-06e6517ee926","Type":"ContainerStarted","Data":"d5149ed885c6386b52ef8b6ccfc3cab22b7d4ae75820f8e332623f5e0272d553"} Oct 30 19:05:44.490758 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.490645 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" event={"ID":"1ac051df-4fd9-441c-af3e-06e6517ee926","Type":"ContainerStarted","Data":"f38704884dd4b8bd41e8a0dba71750ac89320b36906a30f1a36f246b63d4b491"} Oct 30 19:05:44.564640 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.564609 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1c909efe-4bc0-46d8-9246-d12a5250d5b6-kserve-provision-location\") pod \"isvc-canary-create-predictor-8b6998fb4-b8wv5\" (UID: \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:05:44.564781 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.564677 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1c909efe-4bc0-46d8-9246-d12a5250d5b6-proxy-tls\") pod \"isvc-canary-create-predictor-8b6998fb4-b8wv5\" (UID: \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:05:44.564781 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.564758 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1c909efe-4bc0-46d8-9246-d12a5250d5b6-cabundle-cert\") pod \"isvc-canary-create-predictor-8b6998fb4-b8wv5\" (UID: \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:05:44.565544 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.565069 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1c909efe-4bc0-46d8-9246-d12a5250d5b6-kserve-provision-location\") pod \"isvc-canary-create-predictor-8b6998fb4-b8wv5\" (UID: \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:05:44.565544 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.565236 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tnck\" (UniqueName: \"kubernetes.io/projected/1c909efe-4bc0-46d8-9246-d12a5250d5b6-kube-api-access-9tnck\") pod \"isvc-canary-create-predictor-8b6998fb4-b8wv5\" (UID: \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:05:44.565544 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.565338 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1c909efe-4bc0-46d8-9246-d12a5250d5b6-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-8b6998fb4-b8wv5\" (UID: \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:05:44.566064 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.566044 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1c909efe-4bc0-46d8-9246-d12a5250d5b6-kube-rbac-proxy-sar-config\") pod \"isvc-canary-create-predictor-8b6998fb4-b8wv5\" (UID: \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:05:44.566297 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.566277 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1c909efe-4bc0-46d8-9246-d12a5250d5b6-cabundle-cert\") pod \"isvc-canary-create-predictor-8b6998fb4-b8wv5\" (UID: \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:05:44.567806 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.567787 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1c909efe-4bc0-46d8-9246-d12a5250d5b6-proxy-tls\") pod \"isvc-canary-create-predictor-8b6998fb4-b8wv5\" (UID: \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:05:44.626579 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.626542 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tnck\" (UniqueName: \"kubernetes.io/projected/1c909efe-4bc0-46d8-9246-d12a5250d5b6-kube-api-access-9tnck\") pod \"isvc-canary-create-predictor-8b6998fb4-b8wv5\" (UID: \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\") " pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:05:44.678283 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:44.678209 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:05:45.117487 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:45.117456 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5"] Oct 30 19:05:45.122887 ip-10-0-132-74 kubenswrapper[5054]: W1030 19:05:45.122864 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c909efe_4bc0_46d8_9246_d12a5250d5b6.slice/crio-639630adc624b39bdf425249886baf18e4f5e35745b576bfddffc88f56c47714 WatchSource:0}: Error finding container 639630adc624b39bdf425249886baf18e4f5e35745b576bfddffc88f56c47714: Status 404 returned error can't find the container with id 639630adc624b39bdf425249886baf18e4f5e35745b576bfddffc88f56c47714 Oct 30 19:05:45.497135 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:45.497048 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" event={"ID":"1c909efe-4bc0-46d8-9246-d12a5250d5b6","Type":"ContainerStarted","Data":"449f30d8749fe01844627abec67da1a171e019804ca032b040ffdb90891faed3"} Oct 30 19:05:45.497135 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:45.497109 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" event={"ID":"1c909efe-4bc0-46d8-9246-d12a5250d5b6","Type":"ContainerStarted","Data":"639630adc624b39bdf425249886baf18e4f5e35745b576bfddffc88f56c47714"} Oct 30 19:05:48.510440 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:48.510363 5054 generic.go:356] "Generic (PLEG): container finished" podID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerID="d5149ed885c6386b52ef8b6ccfc3cab22b7d4ae75820f8e332623f5e0272d553" exitCode=0 Oct 30 19:05:48.510794 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:48.510436 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" event={"ID":"1ac051df-4fd9-441c-af3e-06e6517ee926","Type":"ContainerDied","Data":"d5149ed885c6386b52ef8b6ccfc3cab22b7d4ae75820f8e332623f5e0272d553"} Oct 30 19:05:49.515268 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:49.515237 5054 generic.go:356] "Generic (PLEG): container finished" podID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerID="449f30d8749fe01844627abec67da1a171e019804ca032b040ffdb90891faed3" exitCode=0 Oct 30 19:05:49.515679 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:49.515317 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" event={"ID":"1c909efe-4bc0-46d8-9246-d12a5250d5b6","Type":"ContainerDied","Data":"449f30d8749fe01844627abec67da1a171e019804ca032b040ffdb90891faed3"} Oct 30 19:05:49.521209 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:49.521179 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" event={"ID":"1ac051df-4fd9-441c-af3e-06e6517ee926","Type":"ContainerStarted","Data":"392f1fc1f80a8965efa53a1147a736a36fdca29f260618aef32bfd497b9aa71e"} Oct 30 19:05:49.521265 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:49.521232 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" event={"ID":"1ac051df-4fd9-441c-af3e-06e6517ee926","Type":"ContainerStarted","Data":"8e6ade030b7ff0c712aa5029cdbcf44920aa9da788bb5599956829351b8f296a"} Oct 30 19:05:49.521501 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:49.521478 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:05:49.521566 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:49.521539 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:05:49.522832 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:49.522810 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" podUID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused" Oct 30 19:05:49.751332 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:49.751273 5054 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" podStartSLOduration=2.718833603 podStartE2EDuration="6.75125256s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.032418957s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:49.750368333 +0000 UTC m=+1691.478747044" watchObservedRunningTime="2026-10-30 19:05:49.75125256 +0000 UTC m=+1691.479631280" Oct 30 19:05:50.526964 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:50.526928 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" event={"ID":"1c909efe-4bc0-46d8-9246-d12a5250d5b6","Type":"ContainerStarted","Data":"840080f014d2916eda1afa5e3025dcb8550ba2ecaf2560b6aac4154c9d5bab21"} Oct 30 19:05:50.527385 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:50.526988 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" event={"ID":"1c909efe-4bc0-46d8-9246-d12a5250d5b6","Type":"ContainerStarted","Data":"ce6507141a20c9ddf91b025435382d95ce531b8e72b94a64cd44a21796a1ae3f"} Oct 30 19:05:50.527385 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:50.527240 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" podUID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused" Oct 30 19:05:50.527530 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:50.527384 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:05:50.645601 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:50.645550 5054 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" podStartSLOduration=2.832072956 podStartE2EDuration="6.645538481s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.813465525s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:50.645061563 +0000 UTC m=+1692.373440285" watchObservedRunningTime="2026-10-30 19:05:50.645538481 +0000 UTC m=+1692.373917196" Oct 30 19:05:51.530964 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:51.530931 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:05:51.532422 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:51.532395 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" podUID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.47:8080: connect: connection refused" Oct 30 19:05:52.535023 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:52.534972 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" podUID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.47:8080: connect: connection refused" Oct 30 19:05:55.532186 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:55.532159 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:05:55.532675 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:55.532647 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" podUID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused" Oct 30 19:05:57.539268 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:57.539240 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:05:57.539680 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:05:57.539654 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" podUID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.47:8080: connect: connection refused" Oct 30 19:06:05.533334 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:06:05.533294 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" podUID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused" Oct 30 19:06:07.539948 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:06:07.539904 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" podUID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.47:8080: connect: connection refused" Oct 30 19:06:08.597424 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:06:08.597382 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:06:15.532627 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:06:15.532582 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" podUID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused" Oct 30 19:06:17.540409 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:06:17.540362 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" podUID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.47:8080: connect: connection refused" Oct 30 19:06:25.532868 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:06:25.532832 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" podUID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused" Oct 30 19:06:27.539960 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:06:27.539913 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" podUID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.47:8080: connect: connection refused" Oct 30 19:06:35.533016 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:06:35.532969 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" podUID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused" Oct 30 19:06:37.540230 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:06:37.540189 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" podUID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.47:8080: connect: connection refused" Oct 30 19:06:38.597591 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:06:38.597558 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:06:45.533129 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:06:45.533092 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" podUID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused" Oct 30 19:06:47.539742 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:06:47.539704 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" podUID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.47:8080: connect: connection refused" Oct 30 19:06:55.532671 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:06:55.532634 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" podUID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused" Oct 30 19:06:57.540210 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:06:57.540175 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" podUID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.47:8080: connect: connection refused" Oct 30 19:07:05.534127 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:05.534096 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:07:06.901624 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:06.901582 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5"] Oct 30 19:07:06.902086 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:06.901927 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" podUID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerName="kserve-container" containerID="cri-o://ce6507141a20c9ddf91b025435382d95ce531b8e72b94a64cd44a21796a1ae3f" gracePeriod=30 Oct 30 19:07:06.902086 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:06.901962 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" podUID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerName="kube-rbac-proxy" containerID="cri-o://840080f014d2916eda1afa5e3025dcb8550ba2ecaf2560b6aac4154c9d5bab21" gracePeriod=30 Oct 30 19:07:06.902996 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:06.902973 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:07:07.536059 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:07.535994 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" podUID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.47:8643/healthz\": dial tcp 10.132.0.47:8643: connect: connection refused" Oct 30 19:07:07.851564 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:07.851490 5054 generic.go:356] "Generic (PLEG): container finished" podID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerID="840080f014d2916eda1afa5e3025dcb8550ba2ecaf2560b6aac4154c9d5bab21" exitCode=0 Oct 30 19:07:07.851564 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:07.851523 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" event={"ID":"1c909efe-4bc0-46d8-9246-d12a5250d5b6","Type":"ContainerDied","Data":"840080f014d2916eda1afa5e3025dcb8550ba2ecaf2560b6aac4154c9d5bab21"} Oct 30 19:07:08.597583 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:07:08.597547 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:07:11.442777 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.442752 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:07:11.592240 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.592151 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c909efe-4bc0-46d8-9246-d12a5250d5b6-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1c909efe-4bc0-46d8-9246-d12a5250d5b6-kube-rbac-proxy-sar-config\") pod \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\" (UID: \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\") " Oct 30 19:07:11.592240 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.592185 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1c909efe-4bc0-46d8-9246-d12a5250d5b6-proxy-tls\" (UniqueName: \"kubernetes.io/secret/1c909efe-4bc0-46d8-9246-d12a5250d5b6-proxy-tls\") pod \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\" (UID: \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\") " Oct 30 19:07:11.592240 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.592222 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c909efe-4bc0-46d8-9246-d12a5250d5b6-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1c909efe-4bc0-46d8-9246-d12a5250d5b6-cabundle-cert\") pod \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\" (UID: \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\") " Oct 30 19:07:11.592512 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.592259 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1c909efe-4bc0-46d8-9246-d12a5250d5b6-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1c909efe-4bc0-46d8-9246-d12a5250d5b6-kserve-provision-location\") pod \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\" (UID: \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\") " Oct 30 19:07:11.592512 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.592286 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c909efe-4bc0-46d8-9246-d12a5250d5b6-kube-api-access-9tnck\" (UniqueName: \"kubernetes.io/projected/1c909efe-4bc0-46d8-9246-d12a5250d5b6-kube-api-access-9tnck\") pod \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\" (UID: \"1c909efe-4bc0-46d8-9246-d12a5250d5b6\") " Oct 30 19:07:11.592621 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.592584 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c909efe-4bc0-46d8-9246-d12a5250d5b6-cabundle-cert" pod "1c909efe-4bc0-46d8-9246-d12a5250d5b6" (UID: "1c909efe-4bc0-46d8-9246-d12a5250d5b6"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:07:11.592621 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.592579 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c909efe-4bc0-46d8-9246-d12a5250d5b6-kube-rbac-proxy-sar-config" pod "1c909efe-4bc0-46d8-9246-d12a5250d5b6" (UID: "1c909efe-4bc0-46d8-9246-d12a5250d5b6"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:07:11.592621 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.592587 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c909efe-4bc0-46d8-9246-d12a5250d5b6-kserve-provision-location" pod "1c909efe-4bc0-46d8-9246-d12a5250d5b6" (UID: "1c909efe-4bc0-46d8-9246-d12a5250d5b6"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:07:11.594415 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.594392 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c909efe-4bc0-46d8-9246-d12a5250d5b6-proxy-tls" pod "1c909efe-4bc0-46d8-9246-d12a5250d5b6" (UID: "1c909efe-4bc0-46d8-9246-d12a5250d5b6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:07:11.594510 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.594498 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c909efe-4bc0-46d8-9246-d12a5250d5b6-kube-api-access-9tnck" pod "1c909efe-4bc0-46d8-9246-d12a5250d5b6" (UID: "1c909efe-4bc0-46d8-9246-d12a5250d5b6"). InnerVolumeSpecName "kube-api-access-9tnck". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:07:11.693078 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.693047 5054 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tnck\" (UniqueName: \"kubernetes.io/projected/1c909efe-4bc0-46d8-9246-d12a5250d5b6-kube-api-access-9tnck\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:07:11.693078 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.693076 5054 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1c909efe-4bc0-46d8-9246-d12a5250d5b6-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:07:11.693260 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.693088 5054 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1c909efe-4bc0-46d8-9246-d12a5250d5b6-proxy-tls\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:07:11.693260 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.693098 5054 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1c909efe-4bc0-46d8-9246-d12a5250d5b6-cabundle-cert\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:07:11.693260 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.693106 5054 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1c909efe-4bc0-46d8-9246-d12a5250d5b6-kserve-provision-location\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:07:11.873110 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.873078 5054 generic.go:356] "Generic (PLEG): container finished" podID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerID="ce6507141a20c9ddf91b025435382d95ce531b8e72b94a64cd44a21796a1ae3f" exitCode=0 Oct 30 19:07:11.873261 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.873138 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" Oct 30 19:07:11.873261 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.873164 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" event={"ID":"1c909efe-4bc0-46d8-9246-d12a5250d5b6","Type":"ContainerDied","Data":"ce6507141a20c9ddf91b025435382d95ce531b8e72b94a64cd44a21796a1ae3f"} Oct 30 19:07:11.873261 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.873225 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5" event={"ID":"1c909efe-4bc0-46d8-9246-d12a5250d5b6","Type":"ContainerDied","Data":"639630adc624b39bdf425249886baf18e4f5e35745b576bfddffc88f56c47714"} Oct 30 19:07:11.873261 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.873255 5054 scope.go:122] "RemoveContainer" containerID="840080f014d2916eda1afa5e3025dcb8550ba2ecaf2560b6aac4154c9d5bab21" Oct 30 19:07:11.881755 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.881740 5054 scope.go:122] "RemoveContainer" containerID="ce6507141a20c9ddf91b025435382d95ce531b8e72b94a64cd44a21796a1ae3f" Oct 30 19:07:11.888994 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.888974 5054 scope.go:122] "RemoveContainer" containerID="449f30d8749fe01844627abec67da1a171e019804ca032b040ffdb90891faed3" Oct 30 19:07:11.895703 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.895685 5054 scope.go:122] "RemoveContainer" containerID="840080f014d2916eda1afa5e3025dcb8550ba2ecaf2560b6aac4154c9d5bab21" Oct 30 19:07:11.895968 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:07:11.895948 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"840080f014d2916eda1afa5e3025dcb8550ba2ecaf2560b6aac4154c9d5bab21\": container with ID starting with 840080f014d2916eda1afa5e3025dcb8550ba2ecaf2560b6aac4154c9d5bab21 not found: ID does not exist" containerID="840080f014d2916eda1afa5e3025dcb8550ba2ecaf2560b6aac4154c9d5bab21" Oct 30 19:07:11.896064 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.895980 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"840080f014d2916eda1afa5e3025dcb8550ba2ecaf2560b6aac4154c9d5bab21"} err="failed to get container status \"840080f014d2916eda1afa5e3025dcb8550ba2ecaf2560b6aac4154c9d5bab21\": rpc error: code = NotFound desc = could not find container \"840080f014d2916eda1afa5e3025dcb8550ba2ecaf2560b6aac4154c9d5bab21\": container with ID starting with 840080f014d2916eda1afa5e3025dcb8550ba2ecaf2560b6aac4154c9d5bab21 not found: ID does not exist" Oct 30 19:07:11.896064 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.896015 5054 scope.go:122] "RemoveContainer" containerID="ce6507141a20c9ddf91b025435382d95ce531b8e72b94a64cd44a21796a1ae3f" Oct 30 19:07:11.896249 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:07:11.896232 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce6507141a20c9ddf91b025435382d95ce531b8e72b94a64cd44a21796a1ae3f\": container with ID starting with ce6507141a20c9ddf91b025435382d95ce531b8e72b94a64cd44a21796a1ae3f not found: ID does not exist" containerID="ce6507141a20c9ddf91b025435382d95ce531b8e72b94a64cd44a21796a1ae3f" Oct 30 19:07:11.896287 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.896257 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce6507141a20c9ddf91b025435382d95ce531b8e72b94a64cd44a21796a1ae3f"} err="failed to get container status \"ce6507141a20c9ddf91b025435382d95ce531b8e72b94a64cd44a21796a1ae3f\": rpc error: code = NotFound desc = could not find container \"ce6507141a20c9ddf91b025435382d95ce531b8e72b94a64cd44a21796a1ae3f\": container with ID starting with ce6507141a20c9ddf91b025435382d95ce531b8e72b94a64cd44a21796a1ae3f not found: ID does not exist" Oct 30 19:07:11.896287 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.896271 5054 scope.go:122] "RemoveContainer" containerID="449f30d8749fe01844627abec67da1a171e019804ca032b040ffdb90891faed3" Oct 30 19:07:11.896473 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:07:11.896456 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"449f30d8749fe01844627abec67da1a171e019804ca032b040ffdb90891faed3\": container with ID starting with 449f30d8749fe01844627abec67da1a171e019804ca032b040ffdb90891faed3 not found: ID does not exist" containerID="449f30d8749fe01844627abec67da1a171e019804ca032b040ffdb90891faed3" Oct 30 19:07:11.896527 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:11.896480 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"449f30d8749fe01844627abec67da1a171e019804ca032b040ffdb90891faed3"} err="failed to get container status \"449f30d8749fe01844627abec67da1a171e019804ca032b040ffdb90891faed3\": rpc error: code = NotFound desc = could not find container \"449f30d8749fe01844627abec67da1a171e019804ca032b040ffdb90891faed3\": container with ID starting with 449f30d8749fe01844627abec67da1a171e019804ca032b040ffdb90891faed3 not found: ID does not exist" Oct 30 19:07:12.037828 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:12.037795 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5"] Oct 30 19:07:12.045803 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:12.045779 5054 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-create-predictor-8b6998fb4-b8wv5"] Oct 30 19:07:12.421709 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:12.421681 5054 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" path="/var/lib/kubelet/pods/1c909efe-4bc0-46d8-9246-d12a5250d5b6/volumes" Oct 30 19:07:23.684849 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:23.684814 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r"] Oct 30 19:07:23.685473 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:23.685058 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" podUID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerName="kserve-container" containerID="cri-o://8e6ade030b7ff0c712aa5029cdbcf44920aa9da788bb5599956829351b8f296a" gracePeriod=30 Oct 30 19:07:23.685473 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:23.685096 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" podUID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerName="kube-rbac-proxy" containerID="cri-o://392f1fc1f80a8965efa53a1147a736a36fdca29f260618aef32bfd497b9aa71e" gracePeriod=30 Oct 30 19:07:23.926062 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:23.926024 5054 generic.go:356] "Generic (PLEG): container finished" podID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerID="392f1fc1f80a8965efa53a1147a736a36fdca29f260618aef32bfd497b9aa71e" exitCode=0 Oct 30 19:07:23.926216 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:23.926028 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" event={"ID":"1ac051df-4fd9-441c-af3e-06e6517ee926","Type":"ContainerDied","Data":"392f1fc1f80a8965efa53a1147a736a36fdca29f260618aef32bfd497b9aa71e"} Oct 30 19:07:25.528475 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:25.528430 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" podUID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.46:8643/healthz\": dial tcp 10.132.0.46:8643: connect: connection refused" Oct 30 19:07:25.532813 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:25.532788 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" podUID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.46:8080: connect: connection refused" Oct 30 19:07:28.226189 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.226167 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:07:28.326277 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.326205 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1ac051df-4fd9-441c-af3e-06e6517ee926-proxy-tls\" (UniqueName: \"kubernetes.io/secret/1ac051df-4fd9-441c-af3e-06e6517ee926-proxy-tls\") pod \"1ac051df-4fd9-441c-af3e-06e6517ee926\" (UID: \"1ac051df-4fd9-441c-af3e-06e6517ee926\") " Oct 30 19:07:28.326277 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.326248 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1ac051df-4fd9-441c-af3e-06e6517ee926-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1ac051df-4fd9-441c-af3e-06e6517ee926-cabundle-cert\") pod \"1ac051df-4fd9-441c-af3e-06e6517ee926\" (UID: \"1ac051df-4fd9-441c-af3e-06e6517ee926\") " Oct 30 19:07:28.326481 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.326304 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1ac051df-4fd9-441c-af3e-06e6517ee926-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1ac051df-4fd9-441c-af3e-06e6517ee926-kube-rbac-proxy-sar-config\") pod \"1ac051df-4fd9-441c-af3e-06e6517ee926\" (UID: \"1ac051df-4fd9-441c-af3e-06e6517ee926\") " Oct 30 19:07:28.326481 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.326326 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ac051df-4fd9-441c-af3e-06e6517ee926-kube-api-access-5pckl\" (UniqueName: \"kubernetes.io/projected/1ac051df-4fd9-441c-af3e-06e6517ee926-kube-api-access-5pckl\") pod \"1ac051df-4fd9-441c-af3e-06e6517ee926\" (UID: \"1ac051df-4fd9-441c-af3e-06e6517ee926\") " Oct 30 19:07:28.326481 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.326349 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ac051df-4fd9-441c-af3e-06e6517ee926-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1ac051df-4fd9-441c-af3e-06e6517ee926-kserve-provision-location\") pod \"1ac051df-4fd9-441c-af3e-06e6517ee926\" (UID: \"1ac051df-4fd9-441c-af3e-06e6517ee926\") " Oct 30 19:07:28.326772 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.326731 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ac051df-4fd9-441c-af3e-06e6517ee926-kube-rbac-proxy-sar-config" pod "1ac051df-4fd9-441c-af3e-06e6517ee926" (UID: "1ac051df-4fd9-441c-af3e-06e6517ee926"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:07:28.326772 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.326740 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ac051df-4fd9-441c-af3e-06e6517ee926-cabundle-cert" pod "1ac051df-4fd9-441c-af3e-06e6517ee926" (UID: "1ac051df-4fd9-441c-af3e-06e6517ee926"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:07:28.326772 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.326743 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ac051df-4fd9-441c-af3e-06e6517ee926-kserve-provision-location" pod "1ac051df-4fd9-441c-af3e-06e6517ee926" (UID: "1ac051df-4fd9-441c-af3e-06e6517ee926"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:07:28.328422 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.328401 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ac051df-4fd9-441c-af3e-06e6517ee926-proxy-tls" pod "1ac051df-4fd9-441c-af3e-06e6517ee926" (UID: "1ac051df-4fd9-441c-af3e-06e6517ee926"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:07:28.328536 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.328517 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ac051df-4fd9-441c-af3e-06e6517ee926-kube-api-access-5pckl" pod "1ac051df-4fd9-441c-af3e-06e6517ee926" (UID: "1ac051df-4fd9-441c-af3e-06e6517ee926"). InnerVolumeSpecName "kube-api-access-5pckl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:07:28.427505 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.427476 5054 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1ac051df-4fd9-441c-af3e-06e6517ee926-cabundle-cert\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:07:28.427505 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.427503 5054 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1ac051df-4fd9-441c-af3e-06e6517ee926-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:07:28.427678 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.427513 5054 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pckl\" (UniqueName: \"kubernetes.io/projected/1ac051df-4fd9-441c-af3e-06e6517ee926-kube-api-access-5pckl\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:07:28.427678 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.427522 5054 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1ac051df-4fd9-441c-af3e-06e6517ee926-kserve-provision-location\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:07:28.427678 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.427532 5054 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1ac051df-4fd9-441c-af3e-06e6517ee926-proxy-tls\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:07:28.947565 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.947532 5054 generic.go:356] "Generic (PLEG): container finished" podID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerID="8e6ade030b7ff0c712aa5029cdbcf44920aa9da788bb5599956829351b8f296a" exitCode=0 Oct 30 19:07:28.947731 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.947566 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" event={"ID":"1ac051df-4fd9-441c-af3e-06e6517ee926","Type":"ContainerDied","Data":"8e6ade030b7ff0c712aa5029cdbcf44920aa9da788bb5599956829351b8f296a"} Oct 30 19:07:28.947731 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.947615 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" Oct 30 19:07:28.947731 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.947669 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r" event={"ID":"1ac051df-4fd9-441c-af3e-06e6517ee926","Type":"ContainerDied","Data":"f38704884dd4b8bd41e8a0dba71750ac89320b36906a30f1a36f246b63d4b491"} Oct 30 19:07:28.947731 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.947708 5054 scope.go:122] "RemoveContainer" containerID="392f1fc1f80a8965efa53a1147a736a36fdca29f260618aef32bfd497b9aa71e" Oct 30 19:07:28.956079 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.956063 5054 scope.go:122] "RemoveContainer" containerID="8e6ade030b7ff0c712aa5029cdbcf44920aa9da788bb5599956829351b8f296a" Oct 30 19:07:28.963427 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.963410 5054 scope.go:122] "RemoveContainer" containerID="d5149ed885c6386b52ef8b6ccfc3cab22b7d4ae75820f8e332623f5e0272d553" Oct 30 19:07:28.970387 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.970369 5054 scope.go:122] "RemoveContainer" containerID="392f1fc1f80a8965efa53a1147a736a36fdca29f260618aef32bfd497b9aa71e" Oct 30 19:07:28.970634 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:07:28.970613 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"392f1fc1f80a8965efa53a1147a736a36fdca29f260618aef32bfd497b9aa71e\": container with ID starting with 392f1fc1f80a8965efa53a1147a736a36fdca29f260618aef32bfd497b9aa71e not found: ID does not exist" containerID="392f1fc1f80a8965efa53a1147a736a36fdca29f260618aef32bfd497b9aa71e" Oct 30 19:07:28.970704 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.970645 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"392f1fc1f80a8965efa53a1147a736a36fdca29f260618aef32bfd497b9aa71e"} err="failed to get container status \"392f1fc1f80a8965efa53a1147a736a36fdca29f260618aef32bfd497b9aa71e\": rpc error: code = NotFound desc = could not find container \"392f1fc1f80a8965efa53a1147a736a36fdca29f260618aef32bfd497b9aa71e\": container with ID starting with 392f1fc1f80a8965efa53a1147a736a36fdca29f260618aef32bfd497b9aa71e not found: ID does not exist" Oct 30 19:07:28.970704 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.970670 5054 scope.go:122] "RemoveContainer" containerID="8e6ade030b7ff0c712aa5029cdbcf44920aa9da788bb5599956829351b8f296a" Oct 30 19:07:28.970906 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:07:28.970886 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e6ade030b7ff0c712aa5029cdbcf44920aa9da788bb5599956829351b8f296a\": container with ID starting with 8e6ade030b7ff0c712aa5029cdbcf44920aa9da788bb5599956829351b8f296a not found: ID does not exist" containerID="8e6ade030b7ff0c712aa5029cdbcf44920aa9da788bb5599956829351b8f296a" Oct 30 19:07:28.970944 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.970913 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e6ade030b7ff0c712aa5029cdbcf44920aa9da788bb5599956829351b8f296a"} err="failed to get container status \"8e6ade030b7ff0c712aa5029cdbcf44920aa9da788bb5599956829351b8f296a\": rpc error: code = NotFound desc = could not find container \"8e6ade030b7ff0c712aa5029cdbcf44920aa9da788bb5599956829351b8f296a\": container with ID starting with 8e6ade030b7ff0c712aa5029cdbcf44920aa9da788bb5599956829351b8f296a not found: ID does not exist" Oct 30 19:07:28.970944 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.970927 5054 scope.go:122] "RemoveContainer" containerID="d5149ed885c6386b52ef8b6ccfc3cab22b7d4ae75820f8e332623f5e0272d553" Oct 30 19:07:28.971192 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:07:28.971174 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5149ed885c6386b52ef8b6ccfc3cab22b7d4ae75820f8e332623f5e0272d553\": container with ID starting with d5149ed885c6386b52ef8b6ccfc3cab22b7d4ae75820f8e332623f5e0272d553 not found: ID does not exist" containerID="d5149ed885c6386b52ef8b6ccfc3cab22b7d4ae75820f8e332623f5e0272d553" Oct 30 19:07:28.971242 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:28.971201 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5149ed885c6386b52ef8b6ccfc3cab22b7d4ae75820f8e332623f5e0272d553"} err="failed to get container status \"d5149ed885c6386b52ef8b6ccfc3cab22b7d4ae75820f8e332623f5e0272d553\": rpc error: code = NotFound desc = could not find container \"d5149ed885c6386b52ef8b6ccfc3cab22b7d4ae75820f8e332623f5e0272d553\": container with ID starting with d5149ed885c6386b52ef8b6ccfc3cab22b7d4ae75820f8e332623f5e0272d553 not found: ID does not exist" Oct 30 19:07:29.158015 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:29.157977 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r"] Oct 30 19:07:29.164383 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:29.164351 5054 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-create-v2-predictor-587db58d45-jtq2r"] Oct 30 19:07:30.421492 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:30.421461 5054 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1ac051df-4fd9-441c-af3e-06e6517ee926" path="/var/lib/kubelet/pods/1ac051df-4fd9-441c-af3e-06e6517ee926/volumes" Oct 30 19:07:32.886522 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.886489 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h"] Oct 30 19:07:32.886903 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.886802 5054 cpu_manager.go:414] "containerMap: removing container" podUID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerName="storage-initializer" Oct 30 19:07:32.886903 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.886812 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerName="storage-initializer" Oct 30 19:07:32.886903 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.886821 5054 cpu_manager.go:414] "containerMap: removing container" podUID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerName="storage-initializer" Oct 30 19:07:32.886903 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.886827 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerName="storage-initializer" Oct 30 19:07:32.886903 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.886837 5054 cpu_manager.go:414] "containerMap: removing container" podUID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerName="kserve-container" Oct 30 19:07:32.886903 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.886842 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerName="kserve-container" Oct 30 19:07:32.886903 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.886850 5054 cpu_manager.go:414] "containerMap: removing container" podUID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerName="kube-rbac-proxy" Oct 30 19:07:32.886903 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.886855 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerName="kube-rbac-proxy" Oct 30 19:07:32.886903 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.886868 5054 cpu_manager.go:414] "containerMap: removing container" podUID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerName="kserve-container" Oct 30 19:07:32.886903 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.886874 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerName="kserve-container" Oct 30 19:07:32.886903 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.886885 5054 cpu_manager.go:414] "containerMap: removing container" podUID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerName="kube-rbac-proxy" Oct 30 19:07:32.886903 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.886890 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerName="kube-rbac-proxy" Oct 30 19:07:32.887292 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.886948 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerName="kube-rbac-proxy" Oct 30 19:07:32.887292 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.886957 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ac051df-4fd9-441c-af3e-06e6517ee926" containerName="kserve-container" Oct 30 19:07:32.887292 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.886964 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerName="kube-rbac-proxy" Oct 30 19:07:32.887292 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.886973 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c909efe-4bc0-46d8-9246-d12a5250d5b6" containerName="kserve-container" Oct 30 19:07:32.890255 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.890233 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:07:32.943714 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.943690 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"kube-root-ca.crt\"" Oct 30 19:07:32.943714 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.943690 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"odh-kserve-custom-ca-bundle\"" Oct 30 19:07:32.943915 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.943690 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"isvc-canary-promote-kube-rbac-proxy-sar-config\"" Oct 30 19:07:32.943915 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.943719 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-gw1-e43a086b9d\"/\"openshift-service-ca.crt\"" Oct 30 19:07:32.991993 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.991967 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"seaweedfs-s3-creds\"" Oct 30 19:07:32.992158 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.991990 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"default-dockercfg-zc4kv\"" Oct 30 19:07:32.992158 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.992154 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"isvc-canary-promote-v2-predictor-serving-cert\"" Oct 30 19:07:32.994285 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:32.994262 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h"] Oct 30 19:07:33.063861 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.063834 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/be2bdc48-665d-48cb-8d32-a6970460c71c-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-9c55d877c-6bp2h\" (UID: \"be2bdc48-665d-48cb-8d32-a6970460c71c\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:07:33.063987 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.063875 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/be2bdc48-665d-48cb-8d32-a6970460c71c-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-9c55d877c-6bp2h\" (UID: \"be2bdc48-665d-48cb-8d32-a6970460c71c\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:07:33.064078 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.063991 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/be2bdc48-665d-48cb-8d32-a6970460c71c-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-9c55d877c-6bp2h\" (UID: \"be2bdc48-665d-48cb-8d32-a6970460c71c\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:07:33.064141 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.064081 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gthcn\" (UniqueName: \"kubernetes.io/projected/be2bdc48-665d-48cb-8d32-a6970460c71c-kube-api-access-gthcn\") pod \"isvc-canary-promote-v2-predictor-9c55d877c-6bp2h\" (UID: \"be2bdc48-665d-48cb-8d32-a6970460c71c\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:07:33.064141 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.064124 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be2bdc48-665d-48cb-8d32-a6970460c71c-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-9c55d877c-6bp2h\" (UID: \"be2bdc48-665d-48cb-8d32-a6970460c71c\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:07:33.165251 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.165179 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/be2bdc48-665d-48cb-8d32-a6970460c71c-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-9c55d877c-6bp2h\" (UID: \"be2bdc48-665d-48cb-8d32-a6970460c71c\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:07:33.165251 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.165246 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/be2bdc48-665d-48cb-8d32-a6970460c71c-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-9c55d877c-6bp2h\" (UID: \"be2bdc48-665d-48cb-8d32-a6970460c71c\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:07:33.165430 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.165262 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gthcn\" (UniqueName: \"kubernetes.io/projected/be2bdc48-665d-48cb-8d32-a6970460c71c-kube-api-access-gthcn\") pod \"isvc-canary-promote-v2-predictor-9c55d877c-6bp2h\" (UID: \"be2bdc48-665d-48cb-8d32-a6970460c71c\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:07:33.165430 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.165283 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be2bdc48-665d-48cb-8d32-a6970460c71c-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-9c55d877c-6bp2h\" (UID: \"be2bdc48-665d-48cb-8d32-a6970460c71c\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:07:33.165430 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.165306 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/be2bdc48-665d-48cb-8d32-a6970460c71c-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-9c55d877c-6bp2h\" (UID: \"be2bdc48-665d-48cb-8d32-a6970460c71c\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:07:33.165666 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.165643 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/be2bdc48-665d-48cb-8d32-a6970460c71c-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-9c55d877c-6bp2h\" (UID: \"be2bdc48-665d-48cb-8d32-a6970460c71c\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:07:33.165979 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.165957 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/be2bdc48-665d-48cb-8d32-a6970460c71c-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-9c55d877c-6bp2h\" (UID: \"be2bdc48-665d-48cb-8d32-a6970460c71c\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:07:33.166078 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.166061 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/be2bdc48-665d-48cb-8d32-a6970460c71c-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-9c55d877c-6bp2h\" (UID: \"be2bdc48-665d-48cb-8d32-a6970460c71c\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:07:33.167855 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.167821 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be2bdc48-665d-48cb-8d32-a6970460c71c-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-9c55d877c-6bp2h\" (UID: \"be2bdc48-665d-48cb-8d32-a6970460c71c\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:07:33.226541 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.226514 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gthcn\" (UniqueName: \"kubernetes.io/projected/be2bdc48-665d-48cb-8d32-a6970460c71c-kube-api-access-gthcn\") pod \"isvc-canary-promote-v2-predictor-9c55d877c-6bp2h\" (UID: \"be2bdc48-665d-48cb-8d32-a6970460c71c\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:07:33.456941 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.456870 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt"] Oct 30 19:07:33.460734 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.460719 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:07:33.507277 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.507251 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:07:33.513510 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.513492 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-gw1-e43a086b9d\"/\"isvc-canary-promote-predictor-serving-cert\"" Oct 30 19:07:33.519757 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.519729 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt"] Oct 30 19:07:33.567553 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.567475 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-659547d46-2pfjt\" (UID: \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:07:33.567553 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.567517 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-cabundle-cert\") pod \"isvc-canary-promote-predictor-659547d46-2pfjt\" (UID: \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:07:33.567730 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.567649 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-proxy-tls\") pod \"isvc-canary-promote-predictor-659547d46-2pfjt\" (UID: \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:07:33.567730 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.567689 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-kserve-provision-location\") pod \"isvc-canary-promote-predictor-659547d46-2pfjt\" (UID: \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:07:33.567730 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.567711 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d78ch\" (UniqueName: \"kubernetes.io/projected/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-kube-api-access-d78ch\") pod \"isvc-canary-promote-predictor-659547d46-2pfjt\" (UID: \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:07:33.668715 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.668678 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-659547d46-2pfjt\" (UID: \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:07:33.668715 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.668719 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-cabundle-cert\") pod \"isvc-canary-promote-predictor-659547d46-2pfjt\" (UID: \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:07:33.668915 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.668759 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-proxy-tls\") pod \"isvc-canary-promote-predictor-659547d46-2pfjt\" (UID: \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:07:33.668915 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.668788 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-kserve-provision-location\") pod \"isvc-canary-promote-predictor-659547d46-2pfjt\" (UID: \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:07:33.668915 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.668807 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d78ch\" (UniqueName: \"kubernetes.io/projected/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-kube-api-access-d78ch\") pod \"isvc-canary-promote-predictor-659547d46-2pfjt\" (UID: \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:07:33.669250 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.669226 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-kserve-provision-location\") pod \"isvc-canary-promote-predictor-659547d46-2pfjt\" (UID: \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:07:33.669450 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.669433 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-predictor-659547d46-2pfjt\" (UID: \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:07:33.669497 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.669462 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-cabundle-cert\") pod \"isvc-canary-promote-predictor-659547d46-2pfjt\" (UID: \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:07:33.671169 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.671149 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-proxy-tls\") pod \"isvc-canary-promote-predictor-659547d46-2pfjt\" (UID: \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:07:33.730617 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.730559 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d78ch\" (UniqueName: \"kubernetes.io/projected/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-kube-api-access-d78ch\") pod \"isvc-canary-promote-predictor-659547d46-2pfjt\" (UID: \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\") " pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:07:33.777034 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.776989 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:07:33.930018 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.929981 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h"] Oct 30 19:07:33.935552 ip-10-0-132-74 kubenswrapper[5054]: W1030 19:07:33.935528 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe2bdc48_665d_48cb_8d32_a6970460c71c.slice/crio-a9478488d1819439ce4a1b1e719ade14e3e00398d9ee90c6f384cbe3dbef7e8f WatchSource:0}: Error finding container a9478488d1819439ce4a1b1e719ade14e3e00398d9ee90c6f384cbe3dbef7e8f: Status 404 returned error can't find the container with id a9478488d1819439ce4a1b1e719ade14e3e00398d9ee90c6f384cbe3dbef7e8f Oct 30 19:07:33.970179 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:33.970144 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" event={"ID":"be2bdc48-665d-48cb-8d32-a6970460c71c","Type":"ContainerStarted","Data":"a9478488d1819439ce4a1b1e719ade14e3e00398d9ee90c6f384cbe3dbef7e8f"} Oct 30 19:07:34.155481 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:34.155447 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt"] Oct 30 19:07:34.161991 ip-10-0-132-74 kubenswrapper[5054]: W1030 19:07:34.161967 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b64a995_c4e6_4256_8b3b_fe39a98e9af7.slice/crio-d89e7c8bc19838d704ec83897c75c6ec663c0aa1001d8c261305bd04804d96ce WatchSource:0}: Error finding container d89e7c8bc19838d704ec83897c75c6ec663c0aa1001d8c261305bd04804d96ce: Status 404 returned error can't find the container with id d89e7c8bc19838d704ec83897c75c6ec663c0aa1001d8c261305bd04804d96ce Oct 30 19:07:34.975314 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:34.975279 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" event={"ID":"1b64a995-c4e6-4256-8b3b-fe39a98e9af7","Type":"ContainerStarted","Data":"81a1a965d39f7e4156777a59ddaf645db2cf5c35c6f69d5af87f6b244d9f8340"} Oct 30 19:07:34.975736 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:34.975340 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" event={"ID":"1b64a995-c4e6-4256-8b3b-fe39a98e9af7","Type":"ContainerStarted","Data":"d89e7c8bc19838d704ec83897c75c6ec663c0aa1001d8c261305bd04804d96ce"} Oct 30 19:07:34.976712 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:34.976688 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" event={"ID":"be2bdc48-665d-48cb-8d32-a6970460c71c","Type":"ContainerStarted","Data":"b042a22cd453e43bac00f0569d801db4cabce350bcd155520f0a04572dd5752a"} Oct 30 19:07:37.990057 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:37.990023 5054 generic.go:356] "Generic (PLEG): container finished" podID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerID="b042a22cd453e43bac00f0569d801db4cabce350bcd155520f0a04572dd5752a" exitCode=0 Oct 30 19:07:37.990411 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:37.990072 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" event={"ID":"be2bdc48-665d-48cb-8d32-a6970460c71c","Type":"ContainerDied","Data":"b042a22cd453e43bac00f0569d801db4cabce350bcd155520f0a04572dd5752a"} Oct 30 19:07:38.573086 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:38.572937 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 19:07:38.599102 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:38.583486 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 19:07:38.607127 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:07:38.607097 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:07:38.996110 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:38.996071 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" event={"ID":"be2bdc48-665d-48cb-8d32-a6970460c71c","Type":"ContainerStarted","Data":"9560d96524fd1b6cd4dea357452a72362e3610b32e25fc28b619327109c6f7ec"} Oct 30 19:07:38.996548 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:38.996168 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" event={"ID":"be2bdc48-665d-48cb-8d32-a6970460c71c","Type":"ContainerStarted","Data":"98af470e2013a2a2a5659ac3b19cf582d8a2ae5d2d1d0996493a16ecb27e1ce5"} Oct 30 19:07:38.996548 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:38.996473 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:07:38.997515 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:38.997490 5054 generic.go:356] "Generic (PLEG): container finished" podID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerID="81a1a965d39f7e4156777a59ddaf645db2cf5c35c6f69d5af87f6b244d9f8340" exitCode=0 Oct 30 19:07:38.997624 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:38.997524 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" event={"ID":"1b64a995-c4e6-4256-8b3b-fe39a98e9af7","Type":"ContainerDied","Data":"81a1a965d39f7e4156777a59ddaf645db2cf5c35c6f69d5af87f6b244d9f8340"} Oct 30 19:07:39.198195 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:39.198141 5054 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" podStartSLOduration=3.314819972 podStartE2EDuration="7.198127252s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.88330728s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:39.163155856 +0000 UTC m=+1800.891534572" watchObservedRunningTime="2026-10-30 19:07:39.198127252 +0000 UTC m=+1800.926505967" Oct 30 19:07:40.003636 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:40.003602 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" event={"ID":"1b64a995-c4e6-4256-8b3b-fe39a98e9af7","Type":"ContainerStarted","Data":"e6cb1b1235fda48dae239e81ebaece2b1d1cd4323bcfcbd7c749e3e97d957bdf"} Oct 30 19:07:40.004090 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:40.003664 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" event={"ID":"1b64a995-c4e6-4256-8b3b-fe39a98e9af7","Type":"ContainerStarted","Data":"f484fb7f5ffaef8ff25d3916b4e85bd4c427c50cdfbb8b46422c947c019e8923"} Oct 30 19:07:40.004090 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:40.003788 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:07:40.004090 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:40.004052 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:07:40.004271 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:40.004187 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:07:40.005222 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:40.005192 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" podUID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.48:8080: connect: connection refused" Oct 30 19:07:40.005331 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:40.005280 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" podUID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.49:8080: connect: connection refused" Oct 30 19:07:40.125826 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:40.125774 5054 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" podStartSLOduration=3.196985824 podStartE2EDuration="7.125760184s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.92877436s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:40.124189269 +0000 UTC m=+1801.852567985" watchObservedRunningTime="2026-10-30 19:07:40.125760184 +0000 UTC m=+1801.854138899" Oct 30 19:07:41.007572 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:41.007528 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" podUID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.49:8080: connect: connection refused" Oct 30 19:07:41.007956 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:41.007633 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" podUID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.48:8080: connect: connection refused" Oct 30 19:07:46.012758 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:46.012728 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:07:46.013226 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:46.012809 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:07:46.013316 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:46.013290 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" podUID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.48:8080: connect: connection refused" Oct 30 19:07:46.013635 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:46.013608 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" podUID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.49:8080: connect: connection refused" Oct 30 19:07:56.013606 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:56.013564 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" podUID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.49:8080: connect: connection refused" Oct 30 19:07:56.014095 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:07:56.013575 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" podUID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.48:8080: connect: connection refused" Oct 30 19:08:06.014038 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:08:06.013981 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" podUID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.49:8080: connect: connection refused" Oct 30 19:08:06.014438 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:08:06.013980 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" podUID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.48:8080: connect: connection refused" Oct 30 19:08:08.597125 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:08:08.597085 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:08:16.013423 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:08:16.013336 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" podUID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.48:8080: connect: connection refused" Oct 30 19:08:16.013814 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:08:16.013477 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" podUID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.49:8080: connect: connection refused" Oct 30 19:08:26.013357 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:08:26.013269 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" podUID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.48:8080: connect: connection refused" Oct 30 19:08:26.013748 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:08:26.013449 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" podUID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.49:8080: connect: connection refused" Oct 30 19:08:36.014218 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:08:36.014180 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" podUID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.49:8080: connect: connection refused" Oct 30 19:08:36.014696 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:08:36.014214 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" podUID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.48:8080: connect: connection refused" Oct 30 19:08:38.596945 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:08:38.596908 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:08:46.013754 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:08:46.013717 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" podUID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.49:8080: connect: connection refused" Oct 30 19:08:46.014192 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:08:46.013723 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" podUID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.48:8080: connect: connection refused" Oct 30 19:08:56.014213 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:08:56.014181 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:08:56.014601 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:08:56.014271 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:09:02.273236 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:02.273196 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt"] Oct 30 19:09:02.273747 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:02.273481 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" podUID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerName="kserve-container" containerID="cri-o://f484fb7f5ffaef8ff25d3916b4e85bd4c427c50cdfbb8b46422c947c019e8923" gracePeriod=30 Oct 30 19:09:02.273747 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:02.273508 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" podUID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerName="kube-rbac-proxy" containerID="cri-o://e6cb1b1235fda48dae239e81ebaece2b1d1cd4323bcfcbd7c749e3e97d957bdf" gracePeriod=30 Oct 30 19:09:03.352016 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:03.351974 5054 generic.go:356] "Generic (PLEG): container finished" podID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerID="e6cb1b1235fda48dae239e81ebaece2b1d1cd4323bcfcbd7c749e3e97d957bdf" exitCode=0 Oct 30 19:09:03.352400 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:03.352051 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" event={"ID":"1b64a995-c4e6-4256-8b3b-fe39a98e9af7","Type":"ContainerDied","Data":"e6cb1b1235fda48dae239e81ebaece2b1d1cd4323bcfcbd7c749e3e97d957bdf"} Oct 30 19:09:06.007715 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:06.007671 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" podUID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.49:8643/healthz\": dial tcp 10.132.0.49:8643: connect: connection refused" Oct 30 19:09:06.013949 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:06.013918 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" podUID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.49:8080: connect: connection refused" Oct 30 19:09:06.825883 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:06.825856 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:09:06.936273 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:06.936197 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-kube-rbac-proxy-sar-config\") pod \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\" (UID: \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\") " Oct 30 19:09:06.936273 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:06.936257 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-cabundle-cert\") pod \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\" (UID: \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\") " Oct 30 19:09:06.936493 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:06.936362 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-kube-api-access-d78ch\" (UniqueName: \"kubernetes.io/projected/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-kube-api-access-d78ch\") pod \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\" (UID: \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\") " Oct 30 19:09:06.936493 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:06.936393 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-proxy-tls\") pod \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\" (UID: \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\") " Oct 30 19:09:06.936493 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:06.936432 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-kserve-provision-location\") pod \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\" (UID: \"1b64a995-c4e6-4256-8b3b-fe39a98e9af7\") " Oct 30 19:09:06.936662 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:06.936592 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-kube-rbac-proxy-sar-config" pod "1b64a995-c4e6-4256-8b3b-fe39a98e9af7" (UID: "1b64a995-c4e6-4256-8b3b-fe39a98e9af7"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:09:06.936662 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:06.936607 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-cabundle-cert" pod "1b64a995-c4e6-4256-8b3b-fe39a98e9af7" (UID: "1b64a995-c4e6-4256-8b3b-fe39a98e9af7"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:09:06.936817 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:06.936793 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-kserve-provision-location" pod "1b64a995-c4e6-4256-8b3b-fe39a98e9af7" (UID: "1b64a995-c4e6-4256-8b3b-fe39a98e9af7"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:09:06.938428 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:06.938399 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-kube-api-access-d78ch" pod "1b64a995-c4e6-4256-8b3b-fe39a98e9af7" (UID: "1b64a995-c4e6-4256-8b3b-fe39a98e9af7"). InnerVolumeSpecName "kube-api-access-d78ch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:09:06.938583 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:06.938563 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-proxy-tls" pod "1b64a995-c4e6-4256-8b3b-fe39a98e9af7" (UID: "1b64a995-c4e6-4256-8b3b-fe39a98e9af7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:09:07.036996 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:07.036967 5054 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-cabundle-cert\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:09:07.036996 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:07.036993 5054 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d78ch\" (UniqueName: \"kubernetes.io/projected/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-kube-api-access-d78ch\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:09:07.037470 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:07.037020 5054 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-proxy-tls\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:09:07.037470 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:07.037031 5054 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-kserve-provision-location\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:09:07.037470 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:07.037040 5054 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1b64a995-c4e6-4256-8b3b-fe39a98e9af7-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:09:07.371803 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:07.371771 5054 generic.go:356] "Generic (PLEG): container finished" podID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerID="f484fb7f5ffaef8ff25d3916b4e85bd4c427c50cdfbb8b46422c947c019e8923" exitCode=0 Oct 30 19:09:07.372014 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:07.371829 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" Oct 30 19:09:07.372014 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:07.371836 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" event={"ID":"1b64a995-c4e6-4256-8b3b-fe39a98e9af7","Type":"ContainerDied","Data":"f484fb7f5ffaef8ff25d3916b4e85bd4c427c50cdfbb8b46422c947c019e8923"} Oct 30 19:09:07.372014 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:07.371913 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt" event={"ID":"1b64a995-c4e6-4256-8b3b-fe39a98e9af7","Type":"ContainerDied","Data":"d89e7c8bc19838d704ec83897c75c6ec663c0aa1001d8c261305bd04804d96ce"} Oct 30 19:09:07.372014 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:07.371953 5054 scope.go:122] "RemoveContainer" containerID="e6cb1b1235fda48dae239e81ebaece2b1d1cd4323bcfcbd7c749e3e97d957bdf" Oct 30 19:09:07.381058 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:07.381039 5054 scope.go:122] "RemoveContainer" containerID="f484fb7f5ffaef8ff25d3916b4e85bd4c427c50cdfbb8b46422c947c019e8923" Oct 30 19:09:07.388346 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:07.388330 5054 scope.go:122] "RemoveContainer" containerID="81a1a965d39f7e4156777a59ddaf645db2cf5c35c6f69d5af87f6b244d9f8340" Oct 30 19:09:07.395573 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:07.395556 5054 scope.go:122] "RemoveContainer" containerID="e6cb1b1235fda48dae239e81ebaece2b1d1cd4323bcfcbd7c749e3e97d957bdf" Oct 30 19:09:07.395820 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:09:07.395801 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6cb1b1235fda48dae239e81ebaece2b1d1cd4323bcfcbd7c749e3e97d957bdf\": container with ID starting with e6cb1b1235fda48dae239e81ebaece2b1d1cd4323bcfcbd7c749e3e97d957bdf not found: ID does not exist" containerID="e6cb1b1235fda48dae239e81ebaece2b1d1cd4323bcfcbd7c749e3e97d957bdf" Oct 30 19:09:07.395908 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:07.395833 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6cb1b1235fda48dae239e81ebaece2b1d1cd4323bcfcbd7c749e3e97d957bdf"} err="failed to get container status \"e6cb1b1235fda48dae239e81ebaece2b1d1cd4323bcfcbd7c749e3e97d957bdf\": rpc error: code = NotFound desc = could not find container \"e6cb1b1235fda48dae239e81ebaece2b1d1cd4323bcfcbd7c749e3e97d957bdf\": container with ID starting with e6cb1b1235fda48dae239e81ebaece2b1d1cd4323bcfcbd7c749e3e97d957bdf not found: ID does not exist" Oct 30 19:09:07.395908 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:07.395857 5054 scope.go:122] "RemoveContainer" containerID="f484fb7f5ffaef8ff25d3916b4e85bd4c427c50cdfbb8b46422c947c019e8923" Oct 30 19:09:07.396117 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:09:07.396099 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f484fb7f5ffaef8ff25d3916b4e85bd4c427c50cdfbb8b46422c947c019e8923\": container with ID starting with f484fb7f5ffaef8ff25d3916b4e85bd4c427c50cdfbb8b46422c947c019e8923 not found: ID does not exist" containerID="f484fb7f5ffaef8ff25d3916b4e85bd4c427c50cdfbb8b46422c947c019e8923" Oct 30 19:09:07.396166 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:07.396122 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f484fb7f5ffaef8ff25d3916b4e85bd4c427c50cdfbb8b46422c947c019e8923"} err="failed to get container status \"f484fb7f5ffaef8ff25d3916b4e85bd4c427c50cdfbb8b46422c947c019e8923\": rpc error: code = NotFound desc = could not find container \"f484fb7f5ffaef8ff25d3916b4e85bd4c427c50cdfbb8b46422c947c019e8923\": container with ID starting with f484fb7f5ffaef8ff25d3916b4e85bd4c427c50cdfbb8b46422c947c019e8923 not found: ID does not exist" Oct 30 19:09:07.396166 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:07.396137 5054 scope.go:122] "RemoveContainer" containerID="81a1a965d39f7e4156777a59ddaf645db2cf5c35c6f69d5af87f6b244d9f8340" Oct 30 19:09:07.396362 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:09:07.396344 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81a1a965d39f7e4156777a59ddaf645db2cf5c35c6f69d5af87f6b244d9f8340\": container with ID starting with 81a1a965d39f7e4156777a59ddaf645db2cf5c35c6f69d5af87f6b244d9f8340 not found: ID does not exist" containerID="81a1a965d39f7e4156777a59ddaf645db2cf5c35c6f69d5af87f6b244d9f8340" Oct 30 19:09:07.396403 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:07.396368 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81a1a965d39f7e4156777a59ddaf645db2cf5c35c6f69d5af87f6b244d9f8340"} err="failed to get container status \"81a1a965d39f7e4156777a59ddaf645db2cf5c35c6f69d5af87f6b244d9f8340\": rpc error: code = NotFound desc = could not find container \"81a1a965d39f7e4156777a59ddaf645db2cf5c35c6f69d5af87f6b244d9f8340\": container with ID starting with 81a1a965d39f7e4156777a59ddaf645db2cf5c35c6f69d5af87f6b244d9f8340 not found: ID does not exist" Oct 30 19:09:07.535547 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:07.535522 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt"] Oct 30 19:09:07.583209 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:07.583185 5054 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-promote-predictor-659547d46-2pfjt"] Oct 30 19:09:08.421208 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:09:08.421176 5054 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" path="/var/lib/kubelet/pods/1b64a995-c4e6-4256-8b3b-fe39a98e9af7/volumes" Oct 30 19:09:08.597179 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:09:08.597142 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:09:38.596188 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:09:38.596153 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:10:08.597762 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:10:08.597719 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:10:38.596278 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:10:38.596237 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:10:43.373958 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:43.373921 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h"] Oct 30 19:10:43.374393 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:43.374190 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" podUID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerName="kserve-container" containerID="cri-o://98af470e2013a2a2a5659ac3b19cf582d8a2ae5d2d1d0996493a16ecb27e1ce5" gracePeriod=30 Oct 30 19:10:43.374393 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:43.374227 5054 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" podUID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerName="kube-rbac-proxy" containerID="cri-o://9560d96524fd1b6cd4dea357452a72362e3610b32e25fc28b619327109c6f7ec" gracePeriod=30 Oct 30 19:10:43.750781 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:43.750699 5054 generic.go:356] "Generic (PLEG): container finished" podID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerID="9560d96524fd1b6cd4dea357452a72362e3610b32e25fc28b619327109c6f7ec" exitCode=0 Oct 30 19:10:43.750781 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:43.750769 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" event={"ID":"be2bdc48-665d-48cb-8d32-a6970460c71c","Type":"ContainerDied","Data":"9560d96524fd1b6cd4dea357452a72362e3610b32e25fc28b619327109c6f7ec"} Oct 30 19:10:46.008501 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:46.008463 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" podUID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.48:8643/healthz\": dial tcp 10.132.0.48:8643: connect: connection refused" Oct 30 19:10:46.013831 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:46.013809 5054 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" podUID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.48:8080: connect: connection refused" Oct 30 19:10:47.727859 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.727832 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:10:47.771195 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.771165 5054 generic.go:356] "Generic (PLEG): container finished" podID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerID="98af470e2013a2a2a5659ac3b19cf582d8a2ae5d2d1d0996493a16ecb27e1ce5" exitCode=0 Oct 30 19:10:47.771313 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.771220 5054 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" Oct 30 19:10:47.771313 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.771252 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" event={"ID":"be2bdc48-665d-48cb-8d32-a6970460c71c","Type":"ContainerDied","Data":"98af470e2013a2a2a5659ac3b19cf582d8a2ae5d2d1d0996493a16ecb27e1ce5"} Oct 30 19:10:47.771313 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.771309 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h" event={"ID":"be2bdc48-665d-48cb-8d32-a6970460c71c","Type":"ContainerDied","Data":"a9478488d1819439ce4a1b1e719ade14e3e00398d9ee90c6f384cbe3dbef7e8f"} Oct 30 19:10:47.771427 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.771336 5054 scope.go:122] "RemoveContainer" containerID="9560d96524fd1b6cd4dea357452a72362e3610b32e25fc28b619327109c6f7ec" Oct 30 19:10:47.779038 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.779022 5054 scope.go:122] "RemoveContainer" containerID="98af470e2013a2a2a5659ac3b19cf582d8a2ae5d2d1d0996493a16ecb27e1ce5" Oct 30 19:10:47.786090 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.786073 5054 scope.go:122] "RemoveContainer" containerID="b042a22cd453e43bac00f0569d801db4cabce350bcd155520f0a04572dd5752a" Oct 30 19:10:47.792895 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.792867 5054 scope.go:122] "RemoveContainer" containerID="9560d96524fd1b6cd4dea357452a72362e3610b32e25fc28b619327109c6f7ec" Oct 30 19:10:47.793157 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:10:47.793137 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9560d96524fd1b6cd4dea357452a72362e3610b32e25fc28b619327109c6f7ec\": container with ID starting with 9560d96524fd1b6cd4dea357452a72362e3610b32e25fc28b619327109c6f7ec not found: ID does not exist" containerID="9560d96524fd1b6cd4dea357452a72362e3610b32e25fc28b619327109c6f7ec" Oct 30 19:10:47.793212 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.793165 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9560d96524fd1b6cd4dea357452a72362e3610b32e25fc28b619327109c6f7ec"} err="failed to get container status \"9560d96524fd1b6cd4dea357452a72362e3610b32e25fc28b619327109c6f7ec\": rpc error: code = NotFound desc = could not find container \"9560d96524fd1b6cd4dea357452a72362e3610b32e25fc28b619327109c6f7ec\": container with ID starting with 9560d96524fd1b6cd4dea357452a72362e3610b32e25fc28b619327109c6f7ec not found: ID does not exist" Oct 30 19:10:47.793212 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.793182 5054 scope.go:122] "RemoveContainer" containerID="98af470e2013a2a2a5659ac3b19cf582d8a2ae5d2d1d0996493a16ecb27e1ce5" Oct 30 19:10:47.793427 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:10:47.793412 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98af470e2013a2a2a5659ac3b19cf582d8a2ae5d2d1d0996493a16ecb27e1ce5\": container with ID starting with 98af470e2013a2a2a5659ac3b19cf582d8a2ae5d2d1d0996493a16ecb27e1ce5 not found: ID does not exist" containerID="98af470e2013a2a2a5659ac3b19cf582d8a2ae5d2d1d0996493a16ecb27e1ce5" Oct 30 19:10:47.793470 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.793432 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98af470e2013a2a2a5659ac3b19cf582d8a2ae5d2d1d0996493a16ecb27e1ce5"} err="failed to get container status \"98af470e2013a2a2a5659ac3b19cf582d8a2ae5d2d1d0996493a16ecb27e1ce5\": rpc error: code = NotFound desc = could not find container \"98af470e2013a2a2a5659ac3b19cf582d8a2ae5d2d1d0996493a16ecb27e1ce5\": container with ID starting with 98af470e2013a2a2a5659ac3b19cf582d8a2ae5d2d1d0996493a16ecb27e1ce5 not found: ID does not exist" Oct 30 19:10:47.793470 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.793445 5054 scope.go:122] "RemoveContainer" containerID="b042a22cd453e43bac00f0569d801db4cabce350bcd155520f0a04572dd5752a" Oct 30 19:10:47.793672 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:10:47.793655 5054 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b042a22cd453e43bac00f0569d801db4cabce350bcd155520f0a04572dd5752a\": container with ID starting with b042a22cd453e43bac00f0569d801db4cabce350bcd155520f0a04572dd5752a not found: ID does not exist" containerID="b042a22cd453e43bac00f0569d801db4cabce350bcd155520f0a04572dd5752a" Oct 30 19:10:47.793717 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.793680 5054 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b042a22cd453e43bac00f0569d801db4cabce350bcd155520f0a04572dd5752a"} err="failed to get container status \"b042a22cd453e43bac00f0569d801db4cabce350bcd155520f0a04572dd5752a\": rpc error: code = NotFound desc = could not find container \"b042a22cd453e43bac00f0569d801db4cabce350bcd155520f0a04572dd5752a\": container with ID starting with b042a22cd453e43bac00f0569d801db4cabce350bcd155520f0a04572dd5752a not found: ID does not exist" Oct 30 19:10:47.820518 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.820476 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/be2bdc48-665d-48cb-8d32-a6970460c71c-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/be2bdc48-665d-48cb-8d32-a6970460c71c-kserve-provision-location\") pod \"be2bdc48-665d-48cb-8d32-a6970460c71c\" (UID: \"be2bdc48-665d-48cb-8d32-a6970460c71c\") " Oct 30 19:10:47.820518 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.820514 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/be2bdc48-665d-48cb-8d32-a6970460c71c-kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/be2bdc48-665d-48cb-8d32-a6970460c71c-kube-rbac-proxy-sar-config\") pod \"be2bdc48-665d-48cb-8d32-a6970460c71c\" (UID: \"be2bdc48-665d-48cb-8d32-a6970460c71c\") " Oct 30 19:10:47.820639 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.820574 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be2bdc48-665d-48cb-8d32-a6970460c71c-kube-api-access-gthcn\" (UniqueName: \"kubernetes.io/projected/be2bdc48-665d-48cb-8d32-a6970460c71c-kube-api-access-gthcn\") pod \"be2bdc48-665d-48cb-8d32-a6970460c71c\" (UID: \"be2bdc48-665d-48cb-8d32-a6970460c71c\") " Oct 30 19:10:47.820639 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.820593 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/be2bdc48-665d-48cb-8d32-a6970460c71c-proxy-tls\" (UniqueName: \"kubernetes.io/secret/be2bdc48-665d-48cb-8d32-a6970460c71c-proxy-tls\") pod \"be2bdc48-665d-48cb-8d32-a6970460c71c\" (UID: \"be2bdc48-665d-48cb-8d32-a6970460c71c\") " Oct 30 19:10:47.820639 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.820616 5054 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/be2bdc48-665d-48cb-8d32-a6970460c71c-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/be2bdc48-665d-48cb-8d32-a6970460c71c-cabundle-cert\") pod \"be2bdc48-665d-48cb-8d32-a6970460c71c\" (UID: \"be2bdc48-665d-48cb-8d32-a6970460c71c\") " Oct 30 19:10:47.820814 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.820790 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be2bdc48-665d-48cb-8d32-a6970460c71c-kserve-provision-location" pod "be2bdc48-665d-48cb-8d32-a6970460c71c" (UID: "be2bdc48-665d-48cb-8d32-a6970460c71c"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:10:47.820978 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.820959 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be2bdc48-665d-48cb-8d32-a6970460c71c-kube-rbac-proxy-sar-config" pod "be2bdc48-665d-48cb-8d32-a6970460c71c" (UID: "be2bdc48-665d-48cb-8d32-a6970460c71c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:10:47.821049 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.821026 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be2bdc48-665d-48cb-8d32-a6970460c71c-cabundle-cert" pod "be2bdc48-665d-48cb-8d32-a6970460c71c" (UID: "be2bdc48-665d-48cb-8d32-a6970460c71c"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:10:47.822456 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.822438 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be2bdc48-665d-48cb-8d32-a6970460c71c-kube-api-access-gthcn" pod "be2bdc48-665d-48cb-8d32-a6970460c71c" (UID: "be2bdc48-665d-48cb-8d32-a6970460c71c"). InnerVolumeSpecName "kube-api-access-gthcn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:10:47.822519 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.822502 5054 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be2bdc48-665d-48cb-8d32-a6970460c71c-proxy-tls" pod "be2bdc48-665d-48cb-8d32-a6970460c71c" (UID: "be2bdc48-665d-48cb-8d32-a6970460c71c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:10:47.921870 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.921843 5054 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/be2bdc48-665d-48cb-8d32-a6970460c71c-kserve-provision-location\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:10:47.921870 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.921866 5054 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/be2bdc48-665d-48cb-8d32-a6970460c71c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:10:47.921870 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.921875 5054 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gthcn\" (UniqueName: \"kubernetes.io/projected/be2bdc48-665d-48cb-8d32-a6970460c71c-kube-api-access-gthcn\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:10:47.922083 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.921885 5054 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/be2bdc48-665d-48cb-8d32-a6970460c71c-proxy-tls\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:10:47.922083 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:47.921894 5054 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/be2bdc48-665d-48cb-8d32-a6970460c71c-cabundle-cert\") on node \"ip-10-0-132-74.us-west-2.compute.internal\" DevicePath \"\"" Oct 30 19:10:48.191081 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:48.191049 5054 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h"] Oct 30 19:10:48.199779 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:48.199756 5054 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-gw1-e43a086b9d/isvc-canary-promote-v2-predictor-9c55d877c-6bp2h"] Oct 30 19:10:48.421092 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:10:48.421062 5054 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be2bdc48-665d-48cb-8d32-a6970460c71c" path="/var/lib/kubelet/pods/be2bdc48-665d-48cb-8d32-a6970460c71c/volumes" Oct 30 19:11:08.597345 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:11:08.597308 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:11:38.596956 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:11:38.596878 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:12:08.597208 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:12:08.597170 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:12:38.597172 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:12:38.597079 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:12:38.601623 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:12:38.601602 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 19:12:38.615799 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:12:38.615781 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 19:13:08.596657 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:13:08.596618 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:13:38.596321 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:13:38.596284 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:14:08.597219 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:14:08.597184 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:14:38.597375 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:14:38.597297 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:15:08.596807 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:15:08.596769 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:15:38.596459 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:15:38.596419 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:16:08.597552 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:16:08.597514 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:16:38.596298 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:16:38.596261 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:17:08.597543 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:17:08.597505 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:17:38.597390 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:17:38.597290 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:17:38.629040 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:17:38.629011 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 19:17:38.640539 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:17:38.640518 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 19:18:08.596933 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:18:08.596895 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:18:38.596904 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:18:38.596865 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:19:08.596702 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:19:08.596664 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:19:38.597153 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:19:38.597116 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:19:55.302285 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:19:55.302258 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pxndl_f1cae505-4cef-46e5-85cc-cf8b410e654a/global-pull-secret-syncer/0.log" Oct 30 19:19:56.466030 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:19:56.465990 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-k4dpj_14b832bf-1a7b-4539-811d-84a11d2b55b6/connection-checker/0.log" Oct 30 19:19:56.872643 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:19:56.872615 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-kbclw_9b5d4a9f-0410-49ba-ab07-c0c69e3fe45f/connection-checker/0.log" Oct 30 19:19:57.278605 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:19:57.278532 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-mzqcg_6050488d-f6ab-4bab-b2b2-90d30f64bab4/connection-checker/0.log" Oct 30 19:19:58.708803 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:19:58.708771 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-w2wtr_1bec85c3-72c2-46dc-9f68-8e80f1478422/konnectivity-agent/0.log" Oct 30 19:19:59.255897 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:19:59.255864 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-74.us-west-2.compute.internal_958b813bafdd4852b0fbfdc2a07cf4ae/haproxy/0.log" Oct 30 19:20:08.596574 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:20:08.596538 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:20:38.596214 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:20:38.596134 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:20:58.182616 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:20:58.182579 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-cf8qt_74153c90-cd72-43b8-85c5-aadb4d66a3e4/cluster-monitoring-operator/0.log" Oct 30 19:21:02.214466 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:02.214439 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mmrxr_c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2/node-exporter/0.log" Oct 30 19:21:02.641939 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:02.641897 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mmrxr_c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2/kube-rbac-proxy/0.log" Oct 30 19:21:03.075609 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:03.075537 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mmrxr_c70d1b0d-1cd4-40c2-8ee9-08f2a07315e2/init-textfile/0.log" Oct 30 19:21:08.344204 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.344173 5054 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc"] Oct 30 19:21:08.344582 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.344475 5054 cpu_manager.go:414] "containerMap: removing container" podUID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerName="storage-initializer" Oct 30 19:21:08.344582 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.344486 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerName="storage-initializer" Oct 30 19:21:08.344582 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.344500 5054 cpu_manager.go:414] "containerMap: removing container" podUID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerName="kserve-container" Oct 30 19:21:08.344582 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.344505 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerName="kserve-container" Oct 30 19:21:08.344582 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.344511 5054 cpu_manager.go:414] "containerMap: removing container" podUID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerName="kube-rbac-proxy" Oct 30 19:21:08.344582 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.344516 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerName="kube-rbac-proxy" Oct 30 19:21:08.344582 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.344526 5054 cpu_manager.go:414] "containerMap: removing container" podUID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerName="kserve-container" Oct 30 19:21:08.344582 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.344530 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerName="kserve-container" Oct 30 19:21:08.344582 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.344542 5054 cpu_manager.go:414] "containerMap: removing container" podUID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerName="storage-initializer" Oct 30 19:21:08.344582 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.344547 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerName="storage-initializer" Oct 30 19:21:08.344582 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.344559 5054 cpu_manager.go:414] "containerMap: removing container" podUID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerName="kube-rbac-proxy" Oct 30 19:21:08.344582 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.344564 5054 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerName="kube-rbac-proxy" Oct 30 19:21:08.344938 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.344623 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerName="kserve-container" Oct 30 19:21:08.344938 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.344630 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="be2bdc48-665d-48cb-8d32-a6970460c71c" containerName="kube-rbac-proxy" Oct 30 19:21:08.344938 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.344637 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerName="kserve-container" Oct 30 19:21:08.344938 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.344644 5054 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b64a995-c4e6-4256-8b3b-fe39a98e9af7" containerName="kube-rbac-proxy" Oct 30 19:21:08.347543 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.347527 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc" Oct 30 19:21:08.402929 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.402905 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc"] Oct 30 19:21:08.453339 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.453311 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-24xkm\"/\"kube-root-ca.crt\"" Oct 30 19:21:08.453479 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.453380 5054 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-24xkm\"/\"openshift-service-ca.crt\"" Oct 30 19:21:08.453479 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.453380 5054 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-24xkm\"/\"default-dockercfg-2tdsl\"" Oct 30 19:21:08.481938 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.481913 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c47a73ff-eb0f-4138-8942-b74eab4beb5c-lib-modules\") pod \"perf-node-gather-daemonset-jn8wc\" (UID: \"c47a73ff-eb0f-4138-8942-b74eab4beb5c\") " pod="openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc" Oct 30 19:21:08.482070 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.481948 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c47a73ff-eb0f-4138-8942-b74eab4beb5c-sys\") pod \"perf-node-gather-daemonset-jn8wc\" (UID: \"c47a73ff-eb0f-4138-8942-b74eab4beb5c\") " pod="openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc" Oct 30 19:21:08.482116 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.482100 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c47a73ff-eb0f-4138-8942-b74eab4beb5c-proc\") pod \"perf-node-gather-daemonset-jn8wc\" (UID: \"c47a73ff-eb0f-4138-8942-b74eab4beb5c\") " pod="openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc" Oct 30 19:21:08.482156 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.482143 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c47a73ff-eb0f-4138-8942-b74eab4beb5c-podres\") pod \"perf-node-gather-daemonset-jn8wc\" (UID: \"c47a73ff-eb0f-4138-8942-b74eab4beb5c\") " pod="openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc" Oct 30 19:21:08.482189 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.482161 5054 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ppm7\" (UniqueName: \"kubernetes.io/projected/c47a73ff-eb0f-4138-8942-b74eab4beb5c-kube-api-access-8ppm7\") pod \"perf-node-gather-daemonset-jn8wc\" (UID: \"c47a73ff-eb0f-4138-8942-b74eab4beb5c\") " pod="openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc" Oct 30 19:21:08.583500 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.583472 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c47a73ff-eb0f-4138-8942-b74eab4beb5c-proc\") pod \"perf-node-gather-daemonset-jn8wc\" (UID: \"c47a73ff-eb0f-4138-8942-b74eab4beb5c\") " pod="openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc" Oct 30 19:21:08.583632 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.583520 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c47a73ff-eb0f-4138-8942-b74eab4beb5c-podres\") pod \"perf-node-gather-daemonset-jn8wc\" (UID: \"c47a73ff-eb0f-4138-8942-b74eab4beb5c\") " pod="openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc" Oct 30 19:21:08.583632 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.583547 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ppm7\" (UniqueName: \"kubernetes.io/projected/c47a73ff-eb0f-4138-8942-b74eab4beb5c-kube-api-access-8ppm7\") pod \"perf-node-gather-daemonset-jn8wc\" (UID: \"c47a73ff-eb0f-4138-8942-b74eab4beb5c\") " pod="openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc" Oct 30 19:21:08.583632 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.583589 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c47a73ff-eb0f-4138-8942-b74eab4beb5c-lib-modules\") pod \"perf-node-gather-daemonset-jn8wc\" (UID: \"c47a73ff-eb0f-4138-8942-b74eab4beb5c\") " pod="openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc" Oct 30 19:21:08.583632 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.583594 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c47a73ff-eb0f-4138-8942-b74eab4beb5c-proc\") pod \"perf-node-gather-daemonset-jn8wc\" (UID: \"c47a73ff-eb0f-4138-8942-b74eab4beb5c\") " pod="openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc" Oct 30 19:21:08.583632 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.583609 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c47a73ff-eb0f-4138-8942-b74eab4beb5c-podres\") pod \"perf-node-gather-daemonset-jn8wc\" (UID: \"c47a73ff-eb0f-4138-8942-b74eab4beb5c\") " pod="openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc" Oct 30 19:21:08.583632 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.583618 5054 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c47a73ff-eb0f-4138-8942-b74eab4beb5c-sys\") pod \"perf-node-gather-daemonset-jn8wc\" (UID: \"c47a73ff-eb0f-4138-8942-b74eab4beb5c\") " pod="openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc" Oct 30 19:21:08.583826 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.583678 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c47a73ff-eb0f-4138-8942-b74eab4beb5c-lib-modules\") pod \"perf-node-gather-daemonset-jn8wc\" (UID: \"c47a73ff-eb0f-4138-8942-b74eab4beb5c\") " pod="openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc" Oct 30 19:21:08.583826 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.583709 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c47a73ff-eb0f-4138-8942-b74eab4beb5c-sys\") pod \"perf-node-gather-daemonset-jn8wc\" (UID: \"c47a73ff-eb0f-4138-8942-b74eab4beb5c\") " pod="openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc" Oct 30 19:21:08.597853 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:21:08.597783 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:21:08.648812 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.648782 5054 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ppm7\" (UniqueName: \"kubernetes.io/projected/c47a73ff-eb0f-4138-8942-b74eab4beb5c-kube-api-access-8ppm7\") pod \"perf-node-gather-daemonset-jn8wc\" (UID: \"c47a73ff-eb0f-4138-8942-b74eab4beb5c\") " pod="openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc" Oct 30 19:21:08.659577 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:08.659556 5054 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc" Oct 30 19:21:09.032424 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:09.032384 5054 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc"] Oct 30 19:21:09.035745 ip-10-0-132-74 kubenswrapper[5054]: W1030 19:21:09.035713 5054 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc47a73ff_eb0f_4138_8942_b74eab4beb5c.slice/crio-b104f93dc5f06799592be9cedc266664e3b33502e5bd4002075ccfd610ee3036 WatchSource:0}: Error finding container b104f93dc5f06799592be9cedc266664e3b33502e5bd4002075ccfd610ee3036: Status 404 returned error can't find the container with id b104f93dc5f06799592be9cedc266664e3b33502e5bd4002075ccfd610ee3036 Oct 30 19:21:09.201384 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:09.201353 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc" event={"ID":"c47a73ff-eb0f-4138-8942-b74eab4beb5c","Type":"ContainerStarted","Data":"8245a1b2262385567bcad0ea6a8f3bed05a240ef1f331e414d92ac4722ca1ec3"} Oct 30 19:21:09.201524 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:09.201408 5054 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc" event={"ID":"c47a73ff-eb0f-4138-8942-b74eab4beb5c","Type":"ContainerStarted","Data":"b104f93dc5f06799592be9cedc266664e3b33502e5bd4002075ccfd610ee3036"} Oct 30 19:21:09.201524 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:09.201476 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc" Oct 30 19:21:09.324198 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:09.324108 5054 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc" podStartSLOduration=1.324096058 podStartE2EDuration="1.324096058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:21:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:09.323337035 +0000 UTC m=+2611.051715751" watchObservedRunningTime="2026-10-30 19:21:09.324096058 +0000 UTC m=+2611.052474774" Oct 30 19:21:10.074591 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:10.074556 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-2dtth_049ceadf-4565-404c-a802-2b8c93770746/prometheus-operator-admission-webhook/0.log" Oct 30 19:21:15.214926 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:15.214901 5054 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-24xkm/perf-node-gather-daemonset-jn8wc" Oct 30 19:21:16.884079 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:16.884050 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-86c478bb88-dl96p_1340fb2d-5f77-4316-a156-f82e0cb1124d/networking-console-plugin/0.log" Oct 30 19:21:19.603764 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:19.603728 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/1.log" Oct 30 19:21:19.608705 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:19.608682 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-tp7sc_0c876fc3-6980-4a99-aed5-7439ca3d6b69/console-operator/2.log" Oct 30 19:21:22.118317 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:22.118291 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5d54c54855-kffjb_9b33fd2f-5244-4df8-9037-b577a80b560c/console/0.log" Oct 30 19:21:22.556889 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:22.556759 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6dd789c8b7-frnlr_4bd54db8-cc7f-4bee-9cb9-99da7a44807c/download-server/0.log" Oct 30 19:21:25.370240 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:25.370211 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-58c9f5f46d-nprgd_2297f719-7e19-4de8-b59c-f75885db9cc6/volume-data-source-validator/0.log" Oct 30 19:21:32.523832 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:32.523799 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ltkzx_447f2b64-f13e-48cc-94b9-e80df4a9d4f3/dns/0.log" Oct 30 19:21:32.947198 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:32.947148 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ltkzx_447f2b64-f13e-48cc-94b9-e80df4a9d4f3/kube-rbac-proxy/0.log" Oct 30 19:21:34.230524 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:34.230477 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k7r7n_d8e44bd9-13d6-4e54-9d9b-bfad33e1393c/dns-node-resolver/0.log" Oct 30 19:21:37.004762 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:37.004729 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6bcc987885-s2p89_89e68c92-6f5f-4be0-9cb1-c78298269a44/registry/0.log" Oct 30 19:21:37.526149 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:37.526117 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mxjc4_6a29b358-3cb7-49e6-85a9-6581a2f52bd1/node-ca/0.log" Oct 30 19:21:38.597247 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:21:38.597212 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:21:42.854753 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:42.854724 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c975679c5-x5lg8_8387e6ab-1e29-4ed8-9ee9-6b9595e4f5d7/router/0.log" Oct 30 19:21:46.224148 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:46.224117 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rbmpn_542d052a-724f-4d98-9be7-7765a8c1d80a/serve-healthcheck-canary/0.log" Oct 30 19:21:48.710331 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:48.710252 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-7c4986d449-q9rgp_00d3be28-5482-48ea-a1be-629e8941236e/insights-operator/0.log" Oct 30 19:21:48.711414 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:48.711372 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-7c4986d449-q9rgp_00d3be28-5482-48ea-a1be-629e8941236e/insights-operator/1.log" Oct 30 19:21:49.130593 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:49.130562 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bscj4_3ccc9d41-f2e7-4b0d-a064-cb3e5549049a/kube-rbac-proxy/0.log" Oct 30 19:21:49.552320 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:49.552238 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bscj4_3ccc9d41-f2e7-4b0d-a064-cb3e5549049a/exporter/0.log" Oct 30 19:21:49.975397 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:49.975371 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bscj4_3ccc9d41-f2e7-4b0d-a064-cb3e5549049a/extractor/0.log" Oct 30 19:21:53.014106 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:21:53.014075 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_periodic-gathering-6x2g6-kxhlf_647f31b7-3d07-41ac-925f-6585e666a757/insights-gathering/0.log" Oct 30 19:22:08.597997 ip-10-0-132-74 kubenswrapper[5054]: E1030 19:22:08.597961 5054 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-bscj4" containerName="extractor" Oct 30 19:22:20.143270 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:22:20.143241 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-7kvml_b4a1d050-5a83-445c-a5a9-03ce4c64cc94/migrator/0.log" Oct 30 19:22:20.566058 ip-10-0-132-74 kubenswrapper[5054]: I1030 19:22:20.565970 5054 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-tsbpt_85fa03e6-458a-45e7-8c7b-a6820618a18a/migrator/0.log"