Oct 30 12:50:59.168349 ip-10-0-141-147 systemd[1]: Starting Kubernetes Kubelet... Oct 30 12:50:59.541489 ip-10-0-141-147 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 12:50:59.541489 ip-10-0-141-147 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 12:50:59.541489 ip-10-0-141-147 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 12:50:59.541489 ip-10-0-141-147 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 12:50:59.545598 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545520 2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 12:50:59.545598 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545597 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 12:50:59.545598 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545601 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 12:50:59.545687 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545605 2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 12:50:59.545687 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545608 2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 12:50:59.545687 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545611 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 12:50:59.545687 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545614 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 12:50:59.545687 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545616 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 12:50:59.545687 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545624 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 12:50:59.545687 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545627 2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 12:50:59.545687 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545630 2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 12:50:59.545687 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545633 2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 12:50:59.545687 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545636 2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 12:50:59.545687 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545639 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 12:50:59.545687 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545642 2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 12:50:59.545687 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545645 2583 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 12:50:59.545687 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545647 2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 12:50:59.545687 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545650 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 12:50:59.545687 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545653 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 12:50:59.545687 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545656 2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 12:50:59.545687 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545658 2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 12:50:59.545687 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545661 2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 12:50:59.545687 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545664 2583 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 12:50:59.546162 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545666 2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 12:50:59.546162 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545669 2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 12:50:59.546162 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545672 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 12:50:59.546162 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545674 2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 12:50:59.546162 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545677 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 12:50:59.546162 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545680 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 12:50:59.546162 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545682 2583 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 12:50:59.546162 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545685 2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 12:50:59.546162 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545688 2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 12:50:59.546162 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545690 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 12:50:59.546162 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545693 2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 12:50:59.546162 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545696 2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 12:50:59.546162 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545699 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 12:50:59.546162 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545701 2583 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 12:50:59.546162 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545704 2583 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 12:50:59.546162 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545707 2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 12:50:59.546162 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545709 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 12:50:59.546162 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545712 2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 12:50:59.546162 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545715 2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 12:50:59.546162 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545717 2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 12:50:59.546839 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545720 2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 12:50:59.546839 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545722 2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 12:50:59.546839 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545725 2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 12:50:59.546839 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545728 2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 12:50:59.546839 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545730 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 12:50:59.546839 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545732 2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 12:50:59.546839 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545735 2583 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 12:50:59.546839 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545737 2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 12:50:59.546839 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545740 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 12:50:59.546839 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545742 2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 12:50:59.546839 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545745 2583 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 12:50:59.546839 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545748 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 12:50:59.546839 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545750 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 12:50:59.546839 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545754 2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 12:50:59.546839 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545757 2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 12:50:59.546839 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545759 2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 12:50:59.546839 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545762 2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 12:50:59.546839 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545764 2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 12:50:59.546839 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545767 2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 12:50:59.547502 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545770 2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 12:50:59.547502 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545772 2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 12:50:59.547502 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545775 2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 12:50:59.547502 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545777 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 12:50:59.547502 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545780 2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 12:50:59.547502 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545782 2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 12:50:59.547502 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545785 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 12:50:59.547502 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545793 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 12:50:59.547502 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545795 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 12:50:59.547502 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545799 2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 12:50:59.547502 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545802 2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 12:50:59.547502 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545804 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 12:50:59.547502 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545807 2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 12:50:59.547502 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545809 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 12:50:59.547502 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545812 2583 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 12:50:59.547502 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545815 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 12:50:59.547502 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545819 2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 12:50:59.547502 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545823 2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 12:50:59.547502 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545827 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 12:50:59.548001 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545830 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 12:50:59.548001 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545832 2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 12:50:59.548001 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545835 2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 12:50:59.548001 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545838 2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 12:50:59.548001 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545840 2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 12:50:59.548001 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545843 2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 12:50:59.548001 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545846 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 12:50:59.548001 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545849 2583 feature_gate.go:386] unrecognized feature gate: Example Oct 30 12:50:59.548001 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545853 2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 12:50:59.548001 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545857 2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 12:50:59.548001 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545860 2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 12:50:59.548001 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545863 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 12:50:59.548001 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545865 2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 12:50:59.548001 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545868 2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 12:50:59.548001 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545871 2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 12:50:59.548001 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545873 2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 12:50:59.548001 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545876 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 12:50:59.548001 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545879 2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 12:50:59.548001 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545881 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 12:50:59.548494 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545884 2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 12:50:59.548494 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545887 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 12:50:59.548494 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545889 2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 12:50:59.548494 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545891 2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 12:50:59.548494 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545894 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 12:50:59.548494 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.545897 2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 12:50:59.548690 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548673 2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 12:50:59.548690 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548689 2583 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 12:50:59.548743 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548693 2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 12:50:59.548743 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548697 2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 12:50:59.548743 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548701 2583 feature_gate.go:386] unrecognized feature gate: Example Oct 30 12:50:59.548743 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548704 2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 12:50:59.548743 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548707 2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 12:50:59.548743 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548710 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 12:50:59.548743 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548713 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 12:50:59.548743 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548715 2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 12:50:59.548743 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548719 2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 12:50:59.548743 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548724 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 12:50:59.548743 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548727 2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 12:50:59.548743 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548730 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 12:50:59.548743 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548733 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 12:50:59.548743 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548736 2583 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 12:50:59.548743 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548739 2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 12:50:59.548743 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548741 2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 12:50:59.548743 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548744 2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 12:50:59.548743 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548747 2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 12:50:59.548743 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548750 2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 12:50:59.549225 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548754 2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 12:50:59.549225 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548756 2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 12:50:59.549225 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548759 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 12:50:59.549225 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548761 2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 12:50:59.549225 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548767 2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 12:50:59.549225 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548770 2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 12:50:59.549225 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548772 2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 12:50:59.549225 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548775 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 12:50:59.549225 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548778 2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 12:50:59.549225 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548781 2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 12:50:59.549225 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548783 2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 12:50:59.549225 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548786 2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 12:50:59.549225 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548788 2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 12:50:59.549225 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548791 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 12:50:59.549225 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548793 2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 12:50:59.549225 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548796 2583 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 12:50:59.549225 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548799 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 12:50:59.549225 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548801 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 12:50:59.549225 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548804 2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 12:50:59.549670 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548806 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 12:50:59.549670 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548808 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 12:50:59.549670 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548811 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 12:50:59.549670 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548813 2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 12:50:59.549670 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548816 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 12:50:59.549670 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548818 2583 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 12:50:59.549670 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548821 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 12:50:59.549670 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548823 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 12:50:59.549670 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548826 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 12:50:59.549670 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548828 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 12:50:59.549670 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548831 2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 12:50:59.549670 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548833 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 12:50:59.549670 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548836 2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 12:50:59.549670 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548839 2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 12:50:59.549670 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548842 2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 12:50:59.549670 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548845 2583 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 12:50:59.549670 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548848 2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 12:50:59.549670 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548850 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 12:50:59.549670 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548853 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 12:50:59.549670 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548855 2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 12:50:59.550155 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548857 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 12:50:59.550155 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548860 2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 12:50:59.550155 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548862 2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 12:50:59.550155 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548865 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 12:50:59.550155 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548868 2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 12:50:59.550155 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548870 2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 12:50:59.550155 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548872 2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 12:50:59.550155 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548875 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 12:50:59.550155 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548877 2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 12:50:59.550155 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548880 2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 12:50:59.550155 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548883 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 12:50:59.550155 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548885 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 12:50:59.550155 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548888 2583 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 12:50:59.550155 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548890 2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 12:50:59.550155 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548893 2583 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 12:50:59.550155 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548895 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 12:50:59.550155 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548899 2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 12:50:59.550155 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548901 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 12:50:59.550155 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548904 2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 12:50:59.550155 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548906 2583 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 12:50:59.550622 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548909 2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 12:50:59.550622 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548911 2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 12:50:59.550622 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548914 2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 12:50:59.550622 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548917 2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 12:50:59.550622 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548919 2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 12:50:59.550622 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548921 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 12:50:59.550622 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548924 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 12:50:59.550622 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548928 2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 12:50:59.550622 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548932 2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 12:50:59.550622 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548936 2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 12:50:59.550622 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548939 2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 12:50:59.550622 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548942 2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 12:50:59.550622 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548945 2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 12:50:59.550622 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548948 2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 12:50:59.550622 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548950 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 12:50:59.550622 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548953 2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 12:50:59.550622 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548955 2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 12:50:59.550622 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548958 2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 12:50:59.550622 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548960 2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 12:50:59.551113 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548963 2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 12:50:59.551113 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548965 2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 12:50:59.551113 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548968 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 12:50:59.551113 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548970 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 12:50:59.551113 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548973 2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 12:50:59.551113 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548976 2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 12:50:59.551113 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.548978 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 12:50:59.551113 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549451 2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 12:50:59.551113 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549457 2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 12:50:59.551113 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549459 2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 12:50:59.551113 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549462 2583 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 12:50:59.551113 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549465 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 12:50:59.551113 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549468 2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 12:50:59.551113 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549470 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 12:50:59.551113 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549473 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 12:50:59.551113 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549475 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 12:50:59.551113 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549478 2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 12:50:59.551113 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549481 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 12:50:59.551113 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549484 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 12:50:59.551113 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549487 2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 12:50:59.551566 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549490 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 12:50:59.551566 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549493 2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 12:50:59.551566 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549496 2583 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 12:50:59.551566 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549498 2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 12:50:59.551566 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549501 2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 12:50:59.551566 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549503 2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 12:50:59.551566 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549506 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 12:50:59.551566 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549509 2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 12:50:59.551566 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549511 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 12:50:59.551566 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549514 2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 12:50:59.551566 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549516 2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 12:50:59.551566 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549519 2583 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 12:50:59.551566 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549521 2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 12:50:59.551566 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549524 2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 12:50:59.551566 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549527 2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 12:50:59.551566 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549529 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 12:50:59.551566 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549533 2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 12:50:59.551566 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549536 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 12:50:59.551566 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549539 2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 12:50:59.552033 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549542 2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 12:50:59.552033 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549544 2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 12:50:59.552033 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549548 2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 12:50:59.552033 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549551 2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 12:50:59.552033 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549553 2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 12:50:59.552033 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549556 2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 12:50:59.552033 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549559 2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 12:50:59.552033 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549562 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 12:50:59.552033 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549564 2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 12:50:59.552033 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549567 2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 12:50:59.552033 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549570 2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 12:50:59.552033 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549572 2583 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 12:50:59.552033 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549575 2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 12:50:59.552033 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549577 2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 12:50:59.552033 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549580 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 12:50:59.552033 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549582 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 12:50:59.552033 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549585 2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 12:50:59.552033 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549587 2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 12:50:59.552033 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549590 2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 12:50:59.552489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549594 2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 12:50:59.552489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549598 2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 12:50:59.552489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549601 2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 12:50:59.552489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549604 2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 12:50:59.552489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549612 2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 12:50:59.552489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549615 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 12:50:59.552489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549617 2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 12:50:59.552489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549620 2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 12:50:59.552489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549622 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 12:50:59.552489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549626 2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 12:50:59.552489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549629 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 12:50:59.552489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549632 2583 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 12:50:59.552489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549634 2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 12:50:59.552489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549637 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 12:50:59.552489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549639 2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 12:50:59.552489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549642 2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 12:50:59.552489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549645 2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 12:50:59.552489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549647 2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 12:50:59.552489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549650 2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 12:50:59.552931 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549653 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 12:50:59.552931 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549656 2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 12:50:59.552931 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549658 2583 feature_gate.go:386] unrecognized feature gate: Example Oct 30 12:50:59.552931 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549661 2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 12:50:59.552931 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549664 2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 12:50:59.552931 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549667 2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 12:50:59.552931 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549670 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 12:50:59.552931 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549672 2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 12:50:59.552931 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549675 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 12:50:59.552931 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549678 2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 12:50:59.552931 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549681 2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 12:50:59.552931 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549683 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 12:50:59.552931 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549686 2583 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 12:50:59.552931 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549688 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 12:50:59.552931 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549691 2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 12:50:59.552931 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549695 2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 12:50:59.552931 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549697 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 12:50:59.552931 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549700 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 12:50:59.552931 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549702 2583 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 12:50:59.552931 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549705 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 12:50:59.553440 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549707 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 12:50:59.553440 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549709 2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 12:50:59.553440 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549712 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 12:50:59.553440 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549714 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 12:50:59.553440 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549717 2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 12:50:59.553440 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549719 2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 12:50:59.553440 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549722 2583 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 12:50:59.553440 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549725 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 12:50:59.553440 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549727 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 12:50:59.553440 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549730 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 12:50:59.553440 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549732 2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 12:50:59.553440 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549735 2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 12:50:59.553440 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549737 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 12:50:59.553440 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549740 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 12:50:59.553440 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549743 2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 12:50:59.553440 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.549745 2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 12:50:59.553440 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550464 2583 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 12:50:59.553440 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550474 2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 12:50:59.553440 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550481 2583 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 12:50:59.553440 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550486 2583 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 12:50:59.553440 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550491 2583 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550494 2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550498 2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550502 2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550505 2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550509 2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550512 2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550515 2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550518 2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550521 2583 flags.go:64] FLAG: --cgroup-root="" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550524 2583 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550527 2583 flags.go:64] FLAG: --client-ca-file="" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550530 2583 flags.go:64] FLAG: --cloud-provider="external" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550533 2583 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550537 2583 flags.go:64] FLAG: --cluster-domain="" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550540 2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550543 2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550546 2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550550 2583 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550554 2583 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550557 2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550560 2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550566 2583 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550569 2583 flags.go:64] FLAG: --contention-profiling="false" Oct 30 12:50:59.553929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550572 2583 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550575 2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550578 2583 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550581 2583 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550585 2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550588 2583 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550591 2583 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550596 2583 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550599 2583 flags.go:64] FLAG: --enable-server="true" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550601 2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550605 2583 flags.go:64] FLAG: --event-burst="100" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550609 2583 flags.go:64] FLAG: --event-qps="50" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550612 2583 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550615 2583 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550617 2583 flags.go:64] FLAG: --eviction-hard="" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550622 2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550625 2583 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550629 2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550632 2583 flags.go:64] FLAG: --eviction-soft="" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550635 2583 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550638 2583 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550641 2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550644 2583 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550647 2583 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550650 2583 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 12:50:59.554513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550653 2583 flags.go:64] FLAG: --feature-gates="" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550658 2583 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550661 2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550664 2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550667 2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550671 2583 flags.go:64] FLAG: --healthz-port="10248" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550675 2583 flags.go:64] FLAG: --help="false" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550678 2583 flags.go:64] FLAG: --hostname-override="ip-10-0-141-147.ec2.internal" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550681 2583 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550684 2583 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550687 2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550690 2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550694 2583 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550696 2583 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550699 2583 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550703 2583 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550706 2583 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550709 2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550712 2583 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550715 2583 flags.go:64] FLAG: --kube-reserved="" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550718 2583 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550721 2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550724 2583 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550727 2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 12:50:59.555139 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550730 2583 flags.go:64] FLAG: --lock-file="" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550732 2583 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550735 2583 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550738 2583 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550744 2583 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550747 2583 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550750 2583 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550752 2583 flags.go:64] FLAG: --logging-format="text" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550756 2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550760 2583 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550762 2583 flags.go:64] FLAG: --manifest-url="" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550765 2583 flags.go:64] FLAG: --manifest-url-header="" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550770 2583 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550774 2583 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550778 2583 flags.go:64] FLAG: --max-pods="110" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550781 2583 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550784 2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550787 2583 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550790 2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550793 2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550796 2583 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550799 2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550806 2583 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550809 2583 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550812 2583 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 12:50:59.555711 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550815 2583 flags.go:64] FLAG: --pod-cidr="" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550818 2583 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550821 2583 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550824 2583 flags.go:64] FLAG: --pods-per-core="0" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550828 2583 flags.go:64] FLAG: --port="10250" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550831 2583 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550833 2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08310a1314a7c0a15" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550837 2583 flags.go:64] FLAG: --qos-reserved="" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550840 2583 flags.go:64] FLAG: --read-only-port="10255" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550843 2583 flags.go:64] FLAG: --register-node="true" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550846 2583 flags.go:64] FLAG: --register-with-taints="" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550849 2583 flags.go:64] FLAG: --registry-burst="10" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550852 2583 flags.go:64] FLAG: --registry-qps="5" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550855 2583 flags.go:64] FLAG: --reserved-cpus="" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550858 2583 flags.go:64] FLAG: --reserved-memory="" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550862 2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550865 2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550868 2583 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550871 2583 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550874 2583 flags.go:64] FLAG: --runonce="false" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550878 2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550881 2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550884 2583 flags.go:64] FLAG: --seccomp-default="false" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550887 2583 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550890 2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550894 2583 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 12:50:59.556324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550897 2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550900 2583 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550903 2583 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550906 2583 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550909 2583 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550912 2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550915 2583 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550918 2583 flags.go:64] FLAG: --system-cgroups="" Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550921 2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550926 2583 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550929 2583 flags.go:64] FLAG: --tls-cert-file="" Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550932 2583 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550936 2583 flags.go:64] FLAG: --tls-min-version="" Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550939 2583 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550942 2583 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550945 2583 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550948 2583 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550951 2583 flags.go:64] FLAG: --v="2" Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550955 2583 flags.go:64] FLAG: --version="false" Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550959 2583 flags.go:64] FLAG: --vmodule="" Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550963 2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.550966 2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551121 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551127 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 12:50:59.556934 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551130 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 12:50:59.557543 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551133 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 12:50:59.557543 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551138 2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 12:50:59.557543 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551141 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 12:50:59.557543 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551143 2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 12:50:59.557543 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551146 2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 12:50:59.557543 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551149 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 12:50:59.557543 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551151 2583 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 12:50:59.557543 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551154 2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 12:50:59.557543 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551157 2583 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 12:50:59.557543 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551159 2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 12:50:59.557543 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551162 2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 12:50:59.557543 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551165 2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 12:50:59.557543 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551167 2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 12:50:59.557543 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551170 2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 12:50:59.557543 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551173 2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 12:50:59.557543 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551176 2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 12:50:59.557543 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551178 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 12:50:59.557543 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551181 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 12:50:59.557543 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551184 2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 12:50:59.557543 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551186 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 12:50:59.558047 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551189 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 12:50:59.558047 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551191 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 12:50:59.558047 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551194 2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 12:50:59.558047 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551196 2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 12:50:59.558047 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551200 2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 12:50:59.558047 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551202 2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 12:50:59.558047 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551205 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 12:50:59.558047 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551208 2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 12:50:59.558047 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551210 2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 12:50:59.558047 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551214 2583 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 12:50:59.558047 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551217 2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 12:50:59.558047 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551220 2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 12:50:59.558047 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551222 2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 12:50:59.558047 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551226 2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 12:50:59.558047 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551229 2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 12:50:59.558047 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551231 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 12:50:59.558047 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551234 2583 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 12:50:59.558047 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551237 2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 12:50:59.558047 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551239 2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 12:50:59.558047 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551242 2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 12:50:59.558535 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551244 2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 12:50:59.558535 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551247 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 12:50:59.558535 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551250 2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 12:50:59.558535 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551253 2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 12:50:59.558535 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551255 2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 12:50:59.558535 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551257 2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 12:50:59.558535 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551260 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 12:50:59.558535 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551263 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 12:50:59.558535 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551266 2583 feature_gate.go:386] unrecognized feature gate: Example Oct 30 12:50:59.558535 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551269 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 12:50:59.558535 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551271 2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 12:50:59.558535 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551273 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 12:50:59.558535 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551277 2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 12:50:59.558535 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551279 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 12:50:59.558535 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551282 2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 12:50:59.558535 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551284 2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 12:50:59.558535 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551287 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 12:50:59.558535 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551289 2583 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 12:50:59.558535 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551292 2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 12:50:59.558984 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551294 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 12:50:59.558984 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551297 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 12:50:59.558984 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551300 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 12:50:59.558984 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551303 2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 12:50:59.558984 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551306 2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 12:50:59.558984 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551308 2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 12:50:59.558984 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551312 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 12:50:59.558984 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551314 2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 12:50:59.558984 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551317 2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 12:50:59.558984 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551319 2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 12:50:59.558984 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551322 2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 12:50:59.558984 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551326 2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 12:50:59.558984 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551331 2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 12:50:59.558984 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551334 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 12:50:59.558984 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551337 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 12:50:59.558984 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551340 2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 12:50:59.558984 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551342 2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 12:50:59.558984 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551345 2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 12:50:59.558984 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551347 2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 12:50:59.559501 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551350 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 12:50:59.559501 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551353 2583 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 12:50:59.559501 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551356 2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 12:50:59.559501 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551358 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 12:50:59.559501 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551361 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 12:50:59.559501 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551364 2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 12:50:59.559501 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551366 2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 12:50:59.559501 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551369 2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 12:50:59.559501 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551372 2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 12:50:59.559501 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551374 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 12:50:59.559501 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551377 2583 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 12:50:59.559501 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551379 2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 12:50:59.559501 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551382 2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 12:50:59.559501 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551385 2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 12:50:59.559501 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551388 2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 12:50:59.559501 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551393 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 12:50:59.559501 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551396 2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 12:50:59.559501 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551399 2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 12:50:59.559501 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551401 2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 12:50:59.559501 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551406 2583 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 12:50:59.559971 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551408 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 12:50:59.559971 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551411 2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 12:50:59.559971 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551414 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 12:50:59.559971 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551416 2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 12:50:59.559971 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551419 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 12:50:59.559971 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.551446 2583 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 12:50:59.559971 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551489 2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 12:50:59.559971 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551494 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 12:50:59.559971 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551497 2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 12:50:59.559971 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551500 2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 12:50:59.559971 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551503 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 12:50:59.559971 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551505 2583 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 12:50:59.559971 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551508 2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 12:50:59.559971 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551511 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 12:50:59.559971 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551513 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 12:50:59.559971 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551516 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 12:50:59.560357 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551519 2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 12:50:59.560357 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551521 2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 12:50:59.560357 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551524 2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 12:50:59.560357 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551527 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 12:50:59.560357 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551529 2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 12:50:59.560357 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551532 2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 12:50:59.560357 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551535 2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 12:50:59.560357 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551537 2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 12:50:59.560357 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551541 2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 12:50:59.560357 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551546 2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 12:50:59.560357 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551549 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 12:50:59.560357 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551553 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 12:50:59.560357 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551556 2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 12:50:59.560357 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551558 2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 12:50:59.560357 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551561 2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 12:50:59.560357 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551564 2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 12:50:59.560357 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551568 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 12:50:59.560357 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551570 2583 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 12:50:59.560357 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551573 2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 12:50:59.560809 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551576 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 12:50:59.560809 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551579 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 12:50:59.560809 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551581 2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 12:50:59.560809 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551584 2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 12:50:59.560809 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551586 2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 12:50:59.560809 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551589 2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 12:50:59.560809 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551592 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 12:50:59.560809 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551594 2583 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 12:50:59.560809 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551597 2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 12:50:59.560809 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551599 2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 12:50:59.560809 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551602 2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 12:50:59.560809 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551606 2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 12:50:59.560809 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551610 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 12:50:59.560809 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551613 2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 12:50:59.560809 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551616 2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 12:50:59.560809 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551619 2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 12:50:59.560809 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551621 2583 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 12:50:59.560809 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551624 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 12:50:59.560809 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551627 2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 12:50:59.561272 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551629 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 12:50:59.561272 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551632 2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 12:50:59.561272 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551634 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 12:50:59.561272 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551637 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 12:50:59.561272 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551640 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 12:50:59.561272 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551643 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 12:50:59.561272 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551646 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 12:50:59.561272 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551649 2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 12:50:59.561272 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551651 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 12:50:59.561272 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551654 2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 12:50:59.561272 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551658 2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 12:50:59.561272 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551660 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 12:50:59.561272 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551664 2583 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 12:50:59.561272 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551666 2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 12:50:59.561272 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551669 2583 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 12:50:59.561272 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551672 2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 12:50:59.561272 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551674 2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 12:50:59.561272 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551677 2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 12:50:59.561272 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551679 2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 12:50:59.561272 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551682 2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 12:50:59.561727 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551684 2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 12:50:59.561727 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551687 2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 12:50:59.561727 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551689 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 12:50:59.561727 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551692 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 12:50:59.561727 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551694 2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 12:50:59.561727 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551697 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 12:50:59.561727 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551700 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 12:50:59.561727 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551703 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 12:50:59.561727 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551705 2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 12:50:59.561727 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551708 2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 12:50:59.561727 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551710 2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 12:50:59.561727 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551713 2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 12:50:59.561727 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551716 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 12:50:59.561727 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551720 2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 12:50:59.561727 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551722 2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 12:50:59.561727 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551725 2583 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 12:50:59.561727 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551727 2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 12:50:59.561727 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551731 2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 12:50:59.561727 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551733 2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 12:50:59.562184 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551736 2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 12:50:59.562184 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551738 2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 12:50:59.562184 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551741 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 12:50:59.562184 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551745 2583 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 12:50:59.562184 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551747 2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 12:50:59.562184 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551750 2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 12:50:59.562184 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551753 2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 12:50:59.562184 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551756 2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 12:50:59.562184 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551758 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 12:50:59.562184 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551761 2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 12:50:59.562184 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551763 2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 12:50:59.562184 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551765 2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 12:50:59.562184 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551768 2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 12:50:59.562184 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551770 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 12:50:59.562184 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551773 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 12:50:59.562184 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551776 2583 feature_gate.go:386] unrecognized feature gate: Example Oct 30 12:50:59.562184 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551778 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 12:50:59.562184 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551781 2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 12:50:59.562184 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.551783 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 12:50:59.562184 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.558650 2583 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 12:50:59.562641 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.558666 2583 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 12:50:59.562641 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.559739 2583 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 12:50:59.562641 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.559747 2583 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 12:50:59.562641 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559803 2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 12:50:59.562641 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559808 2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 12:50:59.562641 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559812 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 12:50:59.562641 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559816 2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 12:50:59.562641 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559818 2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 12:50:59.562641 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559821 2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 12:50:59.562641 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559824 2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 12:50:59.562641 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559827 2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 12:50:59.562641 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559831 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 12:50:59.562641 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559833 2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 12:50:59.562641 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559836 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 12:50:59.562641 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559839 2583 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 12:50:59.562641 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559842 2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 12:50:59.562641 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559844 2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 12:50:59.562641 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559847 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 12:50:59.562641 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559850 2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 12:50:59.563151 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559852 2583 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 12:50:59.563151 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559855 2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 12:50:59.563151 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559857 2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 12:50:59.563151 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559860 2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 12:50:59.563151 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559862 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 12:50:59.563151 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559865 2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 12:50:59.563151 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559868 2583 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 12:50:59.563151 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559870 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 12:50:59.563151 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559873 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 12:50:59.563151 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559877 2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 12:50:59.563151 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559880 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 12:50:59.563151 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559883 2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 12:50:59.563151 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559887 2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 12:50:59.563151 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559891 2583 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 12:50:59.563151 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559893 2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 12:50:59.563151 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559896 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 12:50:59.563151 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559899 2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 12:50:59.563151 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559901 2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 12:50:59.563151 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559904 2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 12:50:59.563599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559907 2583 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 12:50:59.563599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559909 2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 12:50:59.563599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559912 2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 12:50:59.563599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559914 2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 12:50:59.563599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559917 2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 12:50:59.563599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559920 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 12:50:59.563599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559923 2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 12:50:59.563599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559925 2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 12:50:59.563599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559928 2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 12:50:59.563599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559930 2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 12:50:59.563599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559933 2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 12:50:59.563599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559936 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 12:50:59.563599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559938 2583 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 12:50:59.563599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559941 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 12:50:59.563599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559943 2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 12:50:59.563599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559946 2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 12:50:59.563599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559948 2583 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 12:50:59.563599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559951 2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 12:50:59.563599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559954 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 12:50:59.563599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559956 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 12:50:59.564061 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559959 2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 12:50:59.564061 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559961 2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 12:50:59.564061 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559964 2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 12:50:59.564061 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559966 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 12:50:59.564061 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559969 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 12:50:59.564061 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559972 2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 12:50:59.564061 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559975 2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 12:50:59.564061 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559978 2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 12:50:59.564061 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559981 2583 feature_gate.go:386] unrecognized feature gate: Example Oct 30 12:50:59.564061 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559984 2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 12:50:59.564061 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559986 2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 12:50:59.564061 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559989 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 12:50:59.564061 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559991 2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 12:50:59.564061 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559994 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 12:50:59.564061 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559997 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 12:50:59.564061 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.559999 2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 12:50:59.564061 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560002 2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 12:50:59.564061 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560005 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 12:50:59.564061 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560007 2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 12:50:59.564061 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560010 2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 12:50:59.564599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560013 2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 12:50:59.564599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560017 2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 12:50:59.564599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560020 2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 12:50:59.564599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560023 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 12:50:59.564599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560026 2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 12:50:59.564599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560029 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 12:50:59.564599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560031 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 12:50:59.564599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560034 2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 12:50:59.564599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560036 2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 12:50:59.564599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560039 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 12:50:59.564599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560041 2583 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 12:50:59.564599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560044 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 12:50:59.564599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560046 2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 12:50:59.564599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560049 2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 12:50:59.564599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560052 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 12:50:59.564599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560054 2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 12:50:59.564599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560057 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 12:50:59.564599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560059 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 12:50:59.564599 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560074 2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 12:50:59.565084 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560078 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 12:50:59.565084 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560081 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 12:50:59.565084 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560084 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 12:50:59.565084 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560087 2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 12:50:59.565084 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560089 2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 12:50:59.565084 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560092 2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 12:50:59.565084 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560094 2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 12:50:59.565084 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560097 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 12:50:59.565084 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560100 2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 12:50:59.565084 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560103 2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 12:50:59.565084 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560105 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 12:50:59.565084 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560108 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 12:50:59.565084 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.560136 2583 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 12:50:59.565084 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560271 2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 12:50:59.565084 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560276 2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 12:50:59.565084 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560279 2583 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 12:50:59.565489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560281 2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 12:50:59.565489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560284 2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 12:50:59.565489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560286 2583 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 12:50:59.565489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560289 2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 12:50:59.565489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560292 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 12:50:59.565489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560294 2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 12:50:59.565489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560297 2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 12:50:59.565489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560299 2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 12:50:59.565489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560302 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 12:50:59.565489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560305 2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 12:50:59.565489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560308 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 12:50:59.565489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560310 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 12:50:59.565489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560314 2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 12:50:59.565489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560318 2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 12:50:59.565489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560321 2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 12:50:59.565489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560325 2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 12:50:59.565489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560328 2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 12:50:59.565489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560330 2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 12:50:59.565489 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560333 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 12:50:59.565939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560335 2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 12:50:59.565939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560338 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 12:50:59.565939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560340 2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 12:50:59.565939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560343 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 12:50:59.565939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560345 2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 12:50:59.565939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560348 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 12:50:59.565939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560350 2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 12:50:59.565939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560353 2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 12:50:59.565939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560356 2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 12:50:59.565939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560359 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 12:50:59.565939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560361 2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 12:50:59.565939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560364 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 12:50:59.565939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560367 2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 12:50:59.565939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560370 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 12:50:59.565939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560372 2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 12:50:59.565939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560375 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 12:50:59.565939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560377 2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 12:50:59.565939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560380 2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 12:50:59.565939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560382 2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 12:50:59.566391 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560385 2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 12:50:59.566391 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560387 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 12:50:59.566391 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560390 2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 12:50:59.566391 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560392 2583 feature_gate.go:386] unrecognized feature gate: Example Oct 30 12:50:59.566391 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560395 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 12:50:59.566391 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560397 2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 12:50:59.566391 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560399 2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 12:50:59.566391 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560402 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 12:50:59.566391 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560404 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 12:50:59.566391 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560407 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 12:50:59.566391 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560409 2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 12:50:59.566391 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560412 2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 12:50:59.566391 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560414 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 12:50:59.566391 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560417 2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 12:50:59.566391 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560420 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 12:50:59.566391 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560422 2583 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 12:50:59.566391 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560425 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 12:50:59.566391 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560428 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 12:50:59.566391 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560430 2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 12:50:59.566861 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560433 2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 12:50:59.566861 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560435 2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 12:50:59.566861 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560438 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 12:50:59.566861 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560441 2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 12:50:59.566861 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560443 2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 12:50:59.566861 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560446 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 12:50:59.566861 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560449 2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 12:50:59.566861 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560452 2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 12:50:59.566861 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560454 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 12:50:59.566861 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560456 2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 12:50:59.566861 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560459 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 12:50:59.566861 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560461 2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 12:50:59.566861 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560464 2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 12:50:59.566861 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560466 2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 12:50:59.566861 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560469 2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 12:50:59.566861 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560471 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 12:50:59.566861 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560474 2583 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 12:50:59.566861 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560476 2583 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 12:50:59.566861 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560479 2583 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 12:50:59.566861 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560482 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 12:50:59.567478 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560484 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 12:50:59.567478 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560487 2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 12:50:59.567478 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560489 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 12:50:59.567478 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560492 2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 12:50:59.567478 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560495 2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 12:50:59.567478 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560498 2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 12:50:59.567478 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560502 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 12:50:59.567478 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560506 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 12:50:59.567478 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560509 2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 12:50:59.567478 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560512 2583 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 12:50:59.567478 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560516 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 12:50:59.567478 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560519 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 12:50:59.567478 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560522 2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 12:50:59.567478 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560524 2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 12:50:59.567478 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560527 2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 12:50:59.567478 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560529 2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 12:50:59.567478 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560532 2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 12:50:59.567478 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560535 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 12:50:59.567478 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560537 2583 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 12:50:59.568011 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560540 2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 12:50:59.568011 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560542 2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 12:50:59.568011 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560545 2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 12:50:59.568011 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560547 2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 12:50:59.568011 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560550 2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 12:50:59.568011 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560553 2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 12:50:59.568011 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560555 2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 12:50:59.568011 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.560578 2583 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 12:50:59.568011 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560609 2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 12:50:59.568011 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560614 2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 12:50:59.568011 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560617 2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 12:50:59.568011 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560620 2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 12:50:59.568011 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560623 2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 12:50:59.568011 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560627 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 12:50:59.568011 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560630 2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 12:50:59.568011 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560632 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 12:50:59.568422 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560635 2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 12:50:59.568422 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560638 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 12:50:59.568422 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560641 2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 12:50:59.568422 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560644 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 12:50:59.568422 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560647 2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 12:50:59.568422 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560651 2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 12:50:59.568422 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560654 2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 12:50:59.568422 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560657 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 12:50:59.568422 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560661 2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 12:50:59.568422 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560664 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 12:50:59.568422 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560666 2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 12:50:59.568422 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560669 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 12:50:59.568422 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560671 2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 12:50:59.568422 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560674 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 12:50:59.568422 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560677 2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 12:50:59.568422 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560679 2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 12:50:59.568422 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560682 2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 12:50:59.568422 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560685 2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 12:50:59.568422 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560687 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 12:50:59.568890 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560690 2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 12:50:59.568890 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560692 2583 feature_gate.go:386] unrecognized feature gate: Example Oct 30 12:50:59.568890 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560695 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 12:50:59.568890 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560698 2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 12:50:59.568890 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560700 2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 12:50:59.568890 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560703 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 12:50:59.568890 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560705 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 12:50:59.568890 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560708 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 12:50:59.568890 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560710 2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 12:50:59.568890 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560713 2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 12:50:59.568890 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560716 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 12:50:59.568890 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560719 2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 12:50:59.568890 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560722 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 12:50:59.568890 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560724 2583 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 12:50:59.568890 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560727 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 12:50:59.568890 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560730 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 12:50:59.568890 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560732 2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 12:50:59.568890 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560735 2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 12:50:59.568890 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560738 2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 12:50:59.569472 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560741 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 12:50:59.569472 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560743 2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 12:50:59.569472 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560748 2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 12:50:59.569472 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560751 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 12:50:59.569472 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560754 2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 12:50:59.569472 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560757 2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 12:50:59.569472 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560759 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 12:50:59.569472 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560762 2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 12:50:59.569472 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560765 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 12:50:59.569472 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560767 2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 12:50:59.569472 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560770 2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 12:50:59.569472 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560772 2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 12:50:59.569472 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560775 2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 12:50:59.569472 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560777 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 12:50:59.569472 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560780 2583 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 12:50:59.569472 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560782 2583 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 12:50:59.569472 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560785 2583 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 12:50:59.569472 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560788 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 12:50:59.569472 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560790 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 12:50:59.569472 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560793 2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 12:50:59.569939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560795 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 12:50:59.569939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560798 2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 12:50:59.569939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560801 2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 12:50:59.569939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560806 2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 12:50:59.569939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560809 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 12:50:59.569939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560812 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 12:50:59.569939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560815 2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 12:50:59.569939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560817 2583 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 12:50:59.569939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560820 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 12:50:59.569939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560823 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 12:50:59.569939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560825 2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 12:50:59.569939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560828 2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 12:50:59.569939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560830 2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 12:50:59.569939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560834 2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 12:50:59.569939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560838 2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 12:50:59.569939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560840 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 12:50:59.569939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560843 2583 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 12:50:59.569939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560846 2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 12:50:59.569939 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560848 2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 12:50:59.570601 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560851 2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 12:50:59.570601 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560853 2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 12:50:59.570601 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560856 2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 12:50:59.570601 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560858 2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 12:50:59.570601 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560861 2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 12:50:59.570601 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560864 2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 12:50:59.570601 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560866 2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 12:50:59.570601 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560869 2583 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 12:50:59.570601 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560871 2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 12:50:59.570601 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560874 2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 12:50:59.570601 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560876 2583 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 12:50:59.570601 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560879 2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 12:50:59.570601 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560882 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 12:50:59.570601 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560884 2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 12:50:59.570601 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560887 2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 12:50:59.570601 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560889 2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 12:50:59.570601 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560893 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 12:50:59.570601 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560895 2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 12:50:59.570601 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560898 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 12:50:59.571355 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560901 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 12:50:59.571355 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:50:59.560905 2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 12:50:59.571355 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.560957 2583 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 12:50:59.571355 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.564425 2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Oct 30 12:50:59.571355 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.565243 2583 server.go:1014] "Starting client certificate rotation" Oct 30 12:50:59.571355 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.565341 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 12:50:59.571355 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.566043 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 12:50:59.571355 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.567301 2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 12:50:59.571355 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.568465 2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 12:50:59.577624 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.577608 2583 log.go:25] "Validated CRI v1 runtime API" Oct 30 12:50:59.580943 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.580928 2583 log.go:25] "Validated CRI v1 image API" Oct 30 12:50:59.581731 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.581707 2583 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 12:50:59.583839 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.583817 2583 fs.go:135] Filesystem UUIDs: map[280c0489-71da-4ac0-a832-7b90884f3ed2:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 95de3539-e8a7-433c-947e-18b76503bfb5:/dev/nvme0n1p3] Oct 30 12:50:59.583907 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.583838 2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Oct 30 12:50:59.589744 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.589601 2583 manager.go:217] Machine: {Timestamp:2026-10-30 12:50:59.587910429 +0000 UTC m=+0.310713393 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099519 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29b418043af34e36e028f2a0b27fb6 SystemUUID:ec29b418-043a-f34e-36e0-28f2a0b27fb6 BootID:b3e9cffb-e49e-42f3-845e-a30a8ecc9933 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f2:26:71:c9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f2:26:71:c9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ce:46:7b:a5:14:90 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 12:50:59.589831 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.589746 2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 12:50:59.589861 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.589848 2583 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 12:50:59.592704 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.592675 2583 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 12:50:59.592830 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.592706 2583 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-147.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 12:50:59.593484 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.593475 2583 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 12:50:59.593520 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.593487 2583 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 12:50:59.593520 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.593511 2583 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 12:50:59.593568 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.593524 2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 12:50:59.594629 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.594619 2583 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 12:50:59.597757 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.597745 2583 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 12:50:59.597867 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.597857 2583 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 12:50:59.598888 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.598879 2583 kubelet.go:493] "Attempting to sync node with API server" Oct 30 12:50:59.598920 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.598892 2583 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 12:50:59.598920 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.598903 2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 12:50:59.598920 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.598911 2583 kubelet.go:405] "Adding apiserver pod source" Oct 30 12:50:59.598920 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.598919 2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 12:50:59.599279 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.599267 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 12:50:59.600944 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.600929 2583 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 12:50:59.608524 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.608511 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 12:50:59.609427 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.609412 2583 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 12:50:59.609472 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.609462 2583 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 12:50:59.609652 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.609641 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 12:50:59.609680 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.609660 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 12:50:59.609680 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.609670 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 12:50:59.609680 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.609678 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 12:50:59.609758 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.609687 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 12:50:59.609758 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.609696 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 12:50:59.610566 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.610555 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 12:50:59.610598 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.610570 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 12:50:59.610598 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.610581 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 12:50:59.610598 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.610590 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 12:50:59.610676 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.610603 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 12:50:59.610676 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.610617 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 12:50:59.611279 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.611268 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 12:50:59.611316 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.611281 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 12:50:59.615730 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.615707 2583 server.go:1263] "Started kubelet" Oct 30 12:50:59.615832 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.615803 2583 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 12:50:59.615902 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.615798 2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 12:50:59.615902 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.615871 2583 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 12:50:59.616481 ip-10-0-141-147 systemd[1]: Started Kubernetes Kubelet. Oct 30 12:50:59.617520 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.617433 2583 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 12:50:59.617937 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.617922 2583 server.go:317] "Adding debug handlers to kubelet server" Oct 30 12:50:59.621033 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:50:59.621012 2583 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Oct 30 12:50:59.624381 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.624365 2583 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 12:50:59.624453 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.624371 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 12:50:59.624453 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:50:59.624430 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-147.ec2.internal\" not found" Oct 30 12:50:59.624868 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.624848 2583 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 12:50:59.624868 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.624867 2583 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 12:50:59.624984 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.624855 2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 12:50:59.625036 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.624983 2583 reconstruct.go:97] "Volume reconstruction finished" Oct 30 12:50:59.625036 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.624990 2583 reconciler.go:29] "Reconciler: start to sync state" Oct 30 12:50:59.625173 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.625157 2583 factory.go:55] Registering systemd factory Oct 30 12:50:59.625215 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.625197 2583 factory.go:223] Registration of the systemd container factory successfully Oct 30 12:50:59.625420 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.625400 2583 factory.go:153] Registering CRI-O factory Oct 30 12:50:59.625420 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.625421 2583 factory.go:223] Registration of the crio container factory successfully Oct 30 12:50:59.625534 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.625470 2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 12:50:59.625534 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.625486 2583 factory.go:103] Registering Raw factory Oct 30 12:50:59.625534 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.625500 2583 manager.go:1199] Started watching for new ooms in manager Oct 30 12:50:59.626444 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.626428 2583 manager.go:319] Starting recovery of all containers Oct 30 12:50:59.638177 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.638158 2583 manager.go:324] Recovery completed Oct 30 12:50:59.641563 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:50:59.641518 2583 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Oct 30 12:50:59.645377 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.645350 2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 12:50:59.647346 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.647320 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasSufficientMemory" Oct 30 12:50:59.647420 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.647359 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 12:50:59.647420 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.647369 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasSufficientPID" Oct 30 12:50:59.647874 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.647860 2583 cpu_manager.go:226] "Starting" policy="none" Oct 30 12:50:59.647874 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.647870 2583 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 12:50:59.647983 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.647885 2583 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 12:50:59.647983 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.647871 2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 12:50:59.648605 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.648591 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasSufficientMemory" Oct 30 12:50:59.648671 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.648619 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 12:50:59.648671 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.648633 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasSufficientPID" Oct 30 12:50:59.649216 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.649203 2583 policy_none.go:50] "Start" Oct 30 12:50:59.649272 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.649222 2583 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 12:50:59.649272 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.649238 2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 12:50:59.650417 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.650311 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-147.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Oct 30 12:50:59.650417 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.650328 2583 policy_none.go:44] "Start" Oct 30 12:50:59.650417 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:50:59.650315 2583 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-147.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Oct 30 12:50:59.651165 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:50:59.650305 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-147.ec2.internal.18e34fa80121c00c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-147.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-147.ec2.internal,},FirstTimestamp:2026-10-30 12:50:59.615678476 +0000 UTC m=+0.338481435,LastTimestamp:2026-10-30 12:50:59.615678476 +0000 UTC m=+0.338481435,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-147.ec2.internal,}" Oct 30 12:50:59.662863 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.662842 2583 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 12:50:59.666599 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:50:59.666534 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-147.ec2.internal.18e34fa80304ed31 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-147.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-147.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-147.ec2.internal,},FirstTimestamp:2026-10-30 12:50:59.647343921 +0000 UTC m=+0.370146879,LastTimestamp:2026-10-30 12:50:59.647343921 +0000 UTC m=+0.370146879,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-147.ec2.internal,}" Oct 30 12:50:59.666762 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:50:59.666739 2583 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-147.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" Oct 30 12:50:59.666819 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:50:59.666717 2583 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" Oct 30 12:50:59.666819 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:50:59.666761 2583 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" Oct 30 12:50:59.686929 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.686895 2583 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 12:50:59.689914 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.689893 2583 manager.go:341] "Starting Device Plugin manager" Oct 30 12:50:59.690773 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:50:59.690757 2583 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 12:50:59.690773 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.690770 2583 server.go:87] "Starting device plugin registration server" Oct 30 12:50:59.690960 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.690949 2583 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 12:50:59.691007 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.690962 2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 12:50:59.691088 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.691059 2583 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 12:50:59.691404 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.691148 2583 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 12:50:59.691404 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.691158 2583 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 12:50:59.691515 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:50:59.691479 2583 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 12:50:59.691515 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:50:59.691513 2583 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-147.ec2.internal\" not found" Oct 30 12:50:59.693836 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:50:59.693774 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-147.ec2.internal.18e34fa803053ace default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-147.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-141-147.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-141-147.ec2.internal,},FirstTimestamp:2026-10-30 12:50:59.64736379 +0000 UTC m=+0.370166749,LastTimestamp:2026-10-30 12:50:59.64736379 +0000 UTC m=+0.370166749,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-147.ec2.internal,}" Oct 30 12:50:59.697592 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.697566 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wlr25" Oct 30 12:50:59.712234 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.712216 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wlr25" Oct 30 12:50:59.713872 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:50:59.713806 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-147.ec2.internal.18e34fa803055da2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-147.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-141-147.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-141-147.ec2.internal,},FirstTimestamp:2026-10-30 12:50:59.647372706 +0000 UTC m=+0.370175664,LastTimestamp:2026-10-30 12:50:59.647372706 +0000 UTC m=+0.370175664,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-147.ec2.internal,}" Oct 30 12:50:59.777107 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.776937 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 12:50:59.778276 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.778259 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 12:50:59.778375 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.778282 2583 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 12:50:59.778375 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.778316 2583 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 12:50:59.778375 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:50:59.778354 2583 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Oct 30 12:50:59.791452 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.791436 2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 12:50:59.791577 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.791543 2583 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 12:50:59.792214 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.792196 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasSufficientMemory" Oct 30 12:50:59.792271 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.792229 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 12:50:59.792271 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.792241 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasSufficientPID" Oct 30 12:50:59.792271 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.792265 2583 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-141-147.ec2.internal" Oct 30 12:50:59.810917 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.810896 2583 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-141-147.ec2.internal" Oct 30 12:50:59.811004 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:50:59.810921 2583 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-147.ec2.internal\": node \"ip-10-0-141-147.ec2.internal\" not found" Oct 30 12:50:59.879248 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.879215 2583 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-147.ec2.internal"] Oct 30 12:50:59.879365 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.879280 2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 12:50:59.880134 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.880119 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasSufficientMemory" Oct 30 12:50:59.880191 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.880159 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 12:50:59.880191 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.880169 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasSufficientPID" Oct 30 12:50:59.881554 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.881541 2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 12:50:59.881706 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.881690 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" Oct 30 12:50:59.881789 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.881728 2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 12:50:59.882363 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.882345 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasSufficientMemory" Oct 30 12:50:59.882444 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.882377 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 12:50:59.882444 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.882396 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasSufficientPID" Oct 30 12:50:59.882444 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.882376 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasSufficientMemory" Oct 30 12:50:59.882551 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.882456 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 12:50:59.882551 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.882469 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasSufficientPID" Oct 30 12:50:59.883602 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.883588 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-147.ec2.internal" Oct 30 12:50:59.883647 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.883614 2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 12:50:59.884502 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.884487 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasSufficientMemory" Oct 30 12:50:59.884564 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.884509 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 12:50:59.884564 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.884519 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasSufficientPID" Oct 30 12:50:59.889146 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:50:59.888935 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-147.ec2.internal\" not found" Oct 30 12:50:59.904536 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:50:59.904517 2583 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-147.ec2.internal\" not found" node="ip-10-0-141-147.ec2.internal" Oct 30 12:50:59.907149 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:50:59.907133 2583 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-147.ec2.internal\" not found" node="ip-10-0-141-147.ec2.internal" Oct 30 12:50:59.927560 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.927542 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3f5d6256264944b543b87edf9d8fc228-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal\" (UID: \"3f5d6256264944b543b87edf9d8fc228\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" Oct 30 12:50:59.927617 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.927566 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f5d6256264944b543b87edf9d8fc228-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal\" (UID: \"3f5d6256264944b543b87edf9d8fc228\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" Oct 30 12:50:59.927617 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:50:59.927591 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7d0024b3080a170827a29027ec64e58f-config\") pod \"kube-apiserver-proxy-ip-10-0-141-147.ec2.internal\" (UID: \"7d0024b3080a170827a29027ec64e58f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-147.ec2.internal" Oct 30 12:50:59.989405 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:50:59.989383 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-147.ec2.internal\" not found" Oct 30 12:51:00.028112 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:00.028087 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3f5d6256264944b543b87edf9d8fc228-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal\" (UID: \"3f5d6256264944b543b87edf9d8fc228\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" Oct 30 12:51:00.028206 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:00.028119 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f5d6256264944b543b87edf9d8fc228-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal\" (UID: \"3f5d6256264944b543b87edf9d8fc228\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" Oct 30 12:51:00.028206 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:00.028143 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7d0024b3080a170827a29027ec64e58f-config\") pod \"kube-apiserver-proxy-ip-10-0-141-147.ec2.internal\" (UID: \"7d0024b3080a170827a29027ec64e58f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-147.ec2.internal" Oct 30 12:51:00.028206 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:00.028179 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3f5d6256264944b543b87edf9d8fc228-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal\" (UID: \"3f5d6256264944b543b87edf9d8fc228\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" Oct 30 12:51:00.028206 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:00.028182 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f5d6256264944b543b87edf9d8fc228-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal\" (UID: \"3f5d6256264944b543b87edf9d8fc228\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" Oct 30 12:51:00.028363 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:00.028249 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7d0024b3080a170827a29027ec64e58f-config\") pod \"kube-apiserver-proxy-ip-10-0-141-147.ec2.internal\" (UID: \"7d0024b3080a170827a29027ec64e58f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-147.ec2.internal" Oct 30 12:51:00.089901 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:00.089852 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-147.ec2.internal\" not found" Oct 30 12:51:00.190586 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:00.190564 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-147.ec2.internal\" not found" Oct 30 12:51:00.209761 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:00.209739 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" Oct 30 12:51:00.211910 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:00.211895 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-147.ec2.internal" Oct 30 12:51:00.291474 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:00.291440 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-147.ec2.internal\" not found" Oct 30 12:51:00.391922 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:00.391865 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-147.ec2.internal\" not found" Oct 30 12:51:00.492330 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:00.492306 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-147.ec2.internal\" not found" Oct 30 12:51:00.566041 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:00.565808 2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Oct 30 12:51:00.566041 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:00.565992 2583 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Oct 30 12:51:00.592425 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:00.592399 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-147.ec2.internal\" not found" Oct 30 12:51:00.693453 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:00.693421 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-147.ec2.internal\" not found" Oct 30 12:51:00.716312 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:00.716266 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 12:46:00 +0000 UTC" deadline="2028-04-25 18:07:59.848032006 +0000 UTC" Oct 30 12:51:00.716312 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:00.716299 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13037h16m59.131737721s" Oct 30 12:51:00.772859 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:51:00.772821 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f5d6256264944b543b87edf9d8fc228.slice/crio-c0b42f36f921b49fec4166c8f47d231869253e78239ef978d91924ed1d90d1ae WatchSource:0}: Error finding container c0b42f36f921b49fec4166c8f47d231869253e78239ef978d91924ed1d90d1ae: Status 404 returned error can't find the container with id c0b42f36f921b49fec4166c8f47d231869253e78239ef978d91924ed1d90d1ae Oct 30 12:51:00.773405 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:51:00.773387 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d0024b3080a170827a29027ec64e58f.slice/crio-31b370d6fd236b02a8a8fbd660474aaa1fa31699add3ccee185e54d698bddad0 WatchSource:0}: Error finding container 31b370d6fd236b02a8a8fbd660474aaa1fa31699add3ccee185e54d698bddad0: Status 404 returned error can't find the container with id 31b370d6fd236b02a8a8fbd660474aaa1fa31699add3ccee185e54d698bddad0 Oct 30 12:51:00.776716 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:00.776699 2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 12:51:00.777855 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:00.777837 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 12:51:00.777926 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:00.777907 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasSufficientMemory" Oct 30 12:51:00.777965 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:00.777949 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 12:51:00.777998 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:00.777964 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasSufficientPID" Oct 30 12:51:00.782222 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:00.782186 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-147.ec2.internal" event={"ID":"7d0024b3080a170827a29027ec64e58f","Type":"ContainerStarted","Data":"31b370d6fd236b02a8a8fbd660474aaa1fa31699add3ccee185e54d698bddad0"} Oct 30 12:51:00.783277 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:00.783254 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" event={"ID":"3f5d6256264944b543b87edf9d8fc228","Type":"ContainerStarted","Data":"c0b42f36f921b49fec4166c8f47d231869253e78239ef978d91924ed1d90d1ae"} Oct 30 12:51:00.794430 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:00.794405 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-147.ec2.internal\" not found" Oct 30 12:51:00.895037 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:00.895008 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-147.ec2.internal\" not found" Oct 30 12:51:00.903957 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:00.903937 2583 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 12:51:00.954057 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:00.953993 2583 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 12:51:00.995130 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:00.995101 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-147.ec2.internal\" not found" Oct 30 12:51:01.095895 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:01.095862 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-147.ec2.internal\" not found" Oct 30 12:51:01.196871 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:01.196836 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-147.ec2.internal\" not found" Oct 30 12:51:01.281893 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.281803 2583 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 12:51:01.325245 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.325202 2583 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" Oct 30 12:51:01.360108 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.360061 2583 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 12:51:01.361126 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.361101 2583 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-147.ec2.internal" Oct 30 12:51:01.383758 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.383730 2583 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 12:51:01.434229 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.434200 2583 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 12:51:01.600610 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.600542 2583 apiserver.go:52] "Watching apiserver" Oct 30 12:51:01.625348 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.625318 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Oct 30 12:51:01.651047 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.651018 2583 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 12:51:01.651491 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.651465 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr","openshift-image-registry/node-ca-vsf9w","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal","kube-system/konnectivity-agent-vjn55","openshift-network-operator/iptables-alerter-2cvn5","openshift-ovn-kubernetes/ovnkube-node-cm78w","kube-system/global-pull-secret-syncer-lf7b2","kube-system/kube-apiserver-proxy-ip-10-0-141-147.ec2.internal","openshift-cluster-node-tuning-operator/tuned-qwj8g","openshift-dns/node-resolver-759xk","openshift-multus/multus-additional-cni-plugins-s2xpm","openshift-multus/multus-hjpbf","openshift-multus/network-metrics-daemon-24p2b","openshift-network-diagnostics/network-check-target-d6rt6"] Oct 30 12:51:01.653357 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.653330 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:01.653486 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:01.653438 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lf7b2" podUID="75bd9ac5-49ff-4c58-85c7-79b181150555" Oct 30 12:51:01.654716 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.654691 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:01.654851 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:01.654759 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24p2b" podUID="16a02457-151b-4e96-8118-e7695190b9be" Oct 30 12:51:01.655952 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.655932 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:01.656028 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:01.656001 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6rt6" podUID="a5428d1a-8c8b-41ce-ad97-a729ec7a2e43" Oct 30 12:51:01.657193 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.657174 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vjn55" Oct 30 12:51:01.659254 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.659231 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2cvn5" Oct 30 12:51:01.660855 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.660835 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.663451 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.663431 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.665558 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.665541 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:01.667985 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.667965 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vsf9w" Oct 30 12:51:01.669925 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.669906 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-759xk" Oct 30 12:51:01.671682 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.671663 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:01.673384 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.673361 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.683739 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.683648 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 12:51:01.683739 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.683718 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-v4xk8\"" Oct 30 12:51:01.684134 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.684108 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-48745\"" Oct 30 12:51:01.684481 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.684466 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-m2lrf\"" Oct 30 12:51:01.684549 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.684532 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9ppdm\"" Oct 30 12:51:01.684606 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.684593 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 12:51:01.684663 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.684644 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 12:51:01.685900 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.685881 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 12:51:01.686293 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.686258 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 12:51:01.686379 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.686307 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 12:51:01.687609 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.687585 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 12:51:01.687609 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.687591 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 12:51:01.687771 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.687644 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 12:51:01.687771 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.687644 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 12:51:01.687771 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.687650 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 12:51:01.687771 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.687636 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 12:51:01.687771 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.687681 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 12:51:01.700456 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.700247 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 12:51:01.700456 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.700292 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 12:51:01.700456 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.700331 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 12:51:01.700456 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.700366 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 12:51:01.700456 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.700422 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 12:51:01.700456 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.700434 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 12:51:01.701261 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.701232 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 12:51:01.701346 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.701302 2583 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 12:51:01.703132 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.703113 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-sfhnn\"" Oct 30 12:51:01.703273 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.703167 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xj7mb\"" Oct 30 12:51:01.704787 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.704236 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 12:51:01.704787 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.704275 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-87w8v\"" Oct 30 12:51:01.704787 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.704287 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qjjfj\"" Oct 30 12:51:01.714726 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.714706 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 12:51:01.714726 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.714719 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 12:51:01.714893 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.714733 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 12:51:01.714893 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.714796 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 12:51:01.714893 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.714720 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 12:51:01.714893 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.714798 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 12:51:01.714893 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.714793 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7j54b\"" Oct 30 12:51:01.715681 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.715659 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jdqsk" Oct 30 12:51:01.726645 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.726442 2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 12:51:01.728867 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.728845 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jdqsk" Oct 30 12:51:01.736977 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.736958 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-sysctl-d\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.737077 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.736991 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-etc-selinux\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:01.737077 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737016 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3162c3d2-e86f-4ae1-8196-0414509fae9e-cni-binary-copy\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.737077 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737044 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-etc-kubernetes\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.737077 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737070 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/75bd9ac5-49ff-4c58-85c7-79b181150555-kubelet-config\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:01.737303 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737150 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs\") pod \"network-metrics-daemon-24p2b\" (UID: \"16a02457-151b-4e96-8118-e7695190b9be\") " pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:01.737303 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737189 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6zpc\" (UniqueName: \"kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc\") pod \"network-check-target-d6rt6\" (UID: \"a5428d1a-8c8b-41ce-ad97-a729ec7a2e43\") " pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:01.737303 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737216 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-run-openvswitch\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.737303 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737245 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.737303 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737270 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-registration-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:01.737303 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737288 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2a12192e-f0c8-4354-b9fa-1b65b04ebc19-serviceca\") pod \"node-ca-vsf9w\" (UID: \"2a12192e-f0c8-4354-b9fa-1b65b04ebc19\") " pod="openshift-image-registry/node-ca-vsf9w" Oct 30 12:51:01.737655 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737305 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d6a61ca-a15f-4742-8e81-6057c23e51c8-host-slash\") pod \"iptables-alerter-2cvn5\" (UID: \"0d6a61ca-a15f-4742-8e81-6057c23e51c8\") " pod="openshift-network-operator/iptables-alerter-2cvn5" Oct 30 12:51:01.737655 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737373 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-slash\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.737655 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737406 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-var-lib-cni-multus\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.737655 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737432 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:01.737655 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737459 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/49b68250-16ee-42f1-9730-e4d326c96cd8-agent-certs\") pod \"konnectivity-agent-vjn55\" (UID: \"49b68250-16ee-42f1-9730-e4d326c96cd8\") " pod="kube-system/konnectivity-agent-vjn55" Oct 30 12:51:01.737655 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737487 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/49b68250-16ee-42f1-9730-e4d326c96cd8-konnectivity-ca\") pod \"konnectivity-agent-vjn55\" (UID: \"49b68250-16ee-42f1-9730-e4d326c96cd8\") " pod="kube-system/konnectivity-agent-vjn55" Oct 30 12:51:01.737655 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737513 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d-hosts-file\") pod \"node-resolver-759xk\" (UID: \"7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d\") " pod="openshift-dns/node-resolver-759xk" Oct 30 12:51:01.737655 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737537 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-cni-netd\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.737655 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737562 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/23d95081-1188-4002-997e-65ca57111fab-ovn-node-metrics-cert\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.737655 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737588 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-lib-modules\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.737655 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737613 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:01.737655 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737642 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-var-lib-cni-bin\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.738280 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737665 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2a12192e-f0c8-4354-b9fa-1b65b04ebc19-host\") pod \"node-ca-vsf9w\" (UID: \"2a12192e-f0c8-4354-b9fa-1b65b04ebc19\") " pod="openshift-image-registry/node-ca-vsf9w" Oct 30 12:51:01.738280 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737698 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-log-socket\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.738280 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737723 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/23d95081-1188-4002-997e-65ca57111fab-env-overrides\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.738280 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737757 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-systemd\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.738280 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737790 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-run-netns\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.738280 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737817 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-var-lib-openvswitch\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.738280 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737845 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/23d95081-1188-4002-997e-65ca57111fab-ovnkube-script-lib\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.738280 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737883 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-var-lib-kubelet\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.738280 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737957 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmqmx\" (UniqueName: \"kubernetes.io/projected/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-kube-api-access-pmqmx\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.738280 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.737994 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9446374-a7ea-4f95-baa4-d6dad6805b65-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:01.738280 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738019 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd65z\" (UniqueName: \"kubernetes.io/projected/e9446374-a7ea-4f95-baa4-d6dad6805b65-kube-api-access-wd65z\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:01.738280 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738052 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-hostroot\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.738280 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738088 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vkl7\" (UniqueName: \"kubernetes.io/projected/3162c3d2-e86f-4ae1-8196-0414509fae9e-kube-api-access-7vkl7\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.738280 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738164 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d6a61ca-a15f-4742-8e81-6057c23e51c8-iptables-alerter-script\") pod \"iptables-alerter-2cvn5\" (UID: \"0d6a61ca-a15f-4742-8e81-6057c23e51c8\") " pod="openshift-network-operator/iptables-alerter-2cvn5" Oct 30 12:51:01.738280 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738197 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-modprobe-d\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.738280 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738234 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-host\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.739048 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738271 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-tuned\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.739048 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738292 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-tmp\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.739048 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738318 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9446374-a7ea-4f95-baa4-d6dad6805b65-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:01.739048 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738344 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-kubelet\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.739048 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738426 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-systemd-units\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.739048 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738454 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-run-ovn\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.739048 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738481 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-kubernetes\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.739048 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738505 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mld9s\" (UniqueName: \"kubernetes.io/projected/2a12192e-f0c8-4354-b9fa-1b65b04ebc19-kube-api-access-mld9s\") pod \"node-ca-vsf9w\" (UID: \"2a12192e-f0c8-4354-b9fa-1b65b04ebc19\") " pod="openshift-image-registry/node-ca-vsf9w" Oct 30 12:51:01.739048 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738532 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-run-systemd\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.739048 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738556 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-sysconfig\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.739048 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738581 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-sysctl-conf\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.739048 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738645 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-socket-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:01.739048 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738675 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-sys-fs\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:01.739048 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738701 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-run-k8s-cni-cncf-io\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.739048 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738743 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dc5nm\" (UniqueName: \"kubernetes.io/projected/16a02457-151b-4e96-8118-e7695190b9be-kube-api-access-dc5nm\") pod \"network-metrics-daemon-24p2b\" (UID: \"16a02457-151b-4e96-8118-e7695190b9be\") " pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:01.739048 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738771 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-run\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.739866 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738796 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9446374-a7ea-4f95-baa4-d6dad6805b65-cnibin\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:01.739866 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738897 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-system-cni-dir\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.739866 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738938 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-cnibin\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.739866 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.738988 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-daemon-config\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.739866 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.739028 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-etc-openvswitch\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.739866 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.739065 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-node-log\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.739866 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.739108 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9446374-a7ea-4f95-baa4-d6dad6805b65-system-cni-dir\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:01.739866 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.739153 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9446374-a7ea-4f95-baa4-d6dad6805b65-cni-binary-copy\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:01.739866 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.739227 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9446374-a7ea-4f95-baa4-d6dad6805b65-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:01.739866 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.739265 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-conf-dir\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.739866 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.739292 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvwvc\" (UniqueName: \"kubernetes.io/projected/7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d-kube-api-access-mvwvc\") pod \"node-resolver-759xk\" (UID: \"7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d\") " pod="openshift-dns/node-resolver-759xk" Oct 30 12:51:01.739866 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.739318 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-run-netns\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.739866 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.739343 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-var-lib-kubelet\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.739866 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.739380 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-cni-bin\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.739866 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.739406 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-cni-dir\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.739866 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.739433 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8hrc\" (UniqueName: \"kubernetes.io/projected/23d95081-1188-4002-997e-65ca57111fab-kube-api-access-j8hrc\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.740503 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.739456 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-sys\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.740503 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.739482 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-socket-dir-parent\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.740503 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.739505 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/75bd9ac5-49ff-4c58-85c7-79b181150555-dbus\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:01.740503 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.739530 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d-tmp-dir\") pod \"node-resolver-759xk\" (UID: \"7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d\") " pod="openshift-dns/node-resolver-759xk" Oct 30 12:51:01.740503 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.739555 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-run-ovn-kubernetes\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.740503 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.739596 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/23d95081-1188-4002-997e-65ca57111fab-ovnkube-config\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.740503 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.739624 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-device-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:01.740503 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.739686 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-997qf\" (UniqueName: \"kubernetes.io/projected/0b1634e9-3028-4b6a-b651-9155018b3c5a-kube-api-access-997qf\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:01.740503 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.739713 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-run-multus-certs\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.740503 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.739740 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5djfp\" (UniqueName: \"kubernetes.io/projected/0d6a61ca-a15f-4742-8e81-6057c23e51c8-kube-api-access-5djfp\") pod \"iptables-alerter-2cvn5\" (UID: \"0d6a61ca-a15f-4742-8e81-6057c23e51c8\") " pod="openshift-network-operator/iptables-alerter-2cvn5" Oct 30 12:51:01.840438 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.840398 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvwvc\" (UniqueName: \"kubernetes.io/projected/7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d-kube-api-access-mvwvc\") pod \"node-resolver-759xk\" (UID: \"7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d\") " pod="openshift-dns/node-resolver-759xk" Oct 30 12:51:01.840438 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.840429 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-run-netns\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.840679 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.840453 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-var-lib-kubelet\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.840679 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.840477 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-cni-bin\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.840679 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.840500 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-cni-dir\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.840679 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.840512 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-run-netns\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.840679 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.840525 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8hrc\" (UniqueName: \"kubernetes.io/projected/23d95081-1188-4002-997e-65ca57111fab-kube-api-access-j8hrc\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.840679 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.840606 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-sys\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.840679 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.840644 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-socket-dir-parent\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.840679 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.840670 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/75bd9ac5-49ff-4c58-85c7-79b181150555-dbus\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:01.841061 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.840693 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d-tmp-dir\") pod \"node-resolver-759xk\" (UID: \"7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d\") " pod="openshift-dns/node-resolver-759xk" Oct 30 12:51:01.841061 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.840720 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-run-ovn-kubernetes\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.841061 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.840755 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/23d95081-1188-4002-997e-65ca57111fab-ovnkube-config\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.841061 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.840767 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-cni-bin\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.841061 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.840828 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-sys\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.841061 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.840848 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-device-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:01.841061 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.840872 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-997qf\" (UniqueName: \"kubernetes.io/projected/0b1634e9-3028-4b6a-b651-9155018b3c5a-kube-api-access-997qf\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:01.841061 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.840894 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-run-multus-certs\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.841061 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.840916 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5djfp\" (UniqueName: \"kubernetes.io/projected/0d6a61ca-a15f-4742-8e81-6057c23e51c8-kube-api-access-5djfp\") pod \"iptables-alerter-2cvn5\" (UID: \"0d6a61ca-a15f-4742-8e81-6057c23e51c8\") " pod="openshift-network-operator/iptables-alerter-2cvn5" Oct 30 12:51:01.841061 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.840930 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-socket-dir-parent\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.841061 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.840955 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-sysctl-d\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.841061 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841001 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-cni-dir\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.841061 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841007 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/75bd9ac5-49ff-4c58-85c7-79b181150555-dbus\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:01.841061 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841004 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-etc-selinux\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:01.841061 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841056 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-var-lib-kubelet\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.841061 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841056 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3162c3d2-e86f-4ae1-8196-0414509fae9e-cni-binary-copy\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.841882 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841101 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-etc-kubernetes\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.841882 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841121 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-run-ovn-kubernetes\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.841882 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841167 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/75bd9ac5-49ff-4c58-85c7-79b181150555-kubelet-config\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:01.841882 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841269 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs\") pod \"network-metrics-daemon-24p2b\" (UID: \"16a02457-151b-4e96-8118-e7695190b9be\") " pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:01.841882 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841298 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6zpc\" (UniqueName: \"kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc\") pod \"network-check-target-d6rt6\" (UID: \"a5428d1a-8c8b-41ce-ad97-a729ec7a2e43\") " pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:01.841882 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841324 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-run-openvswitch\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.841882 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841355 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.841882 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841377 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d-tmp-dir\") pod \"node-resolver-759xk\" (UID: \"7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d\") " pod="openshift-dns/node-resolver-759xk" Oct 30 12:51:01.841882 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841383 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-registration-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:01.841882 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841413 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2a12192e-f0c8-4354-b9fa-1b65b04ebc19-serviceca\") pod \"node-ca-vsf9w\" (UID: \"2a12192e-f0c8-4354-b9fa-1b65b04ebc19\") " pod="openshift-image-registry/node-ca-vsf9w" Oct 30 12:51:01.841882 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841730 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3162c3d2-e86f-4ae1-8196-0414509fae9e-cni-binary-copy\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.841882 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841751 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d6a61ca-a15f-4742-8e81-6057c23e51c8-host-slash\") pod \"iptables-alerter-2cvn5\" (UID: \"0d6a61ca-a15f-4742-8e81-6057c23e51c8\") " pod="openshift-network-operator/iptables-alerter-2cvn5" Oct 30 12:51:01.841882 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841843 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/75bd9ac5-49ff-4c58-85c7-79b181150555-kubelet-config\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:01.841882 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841871 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-etc-kubernetes\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.841882 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841878 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.842593 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841899 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-run-openvswitch\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.842593 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841902 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-slash\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.842593 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841931 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2a12192e-f0c8-4354-b9fa-1b65b04ebc19-serviceca\") pod \"node-ca-vsf9w\" (UID: \"2a12192e-f0c8-4354-b9fa-1b65b04ebc19\") " pod="openshift-image-registry/node-ca-vsf9w" Oct 30 12:51:01.842593 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841946 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-run-multus-certs\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.842593 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841936 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d6a61ca-a15f-4742-8e81-6057c23e51c8-host-slash\") pod \"iptables-alerter-2cvn5\" (UID: \"0d6a61ca-a15f-4742-8e81-6057c23e51c8\") " pod="openshift-network-operator/iptables-alerter-2cvn5" Oct 30 12:51:01.842593 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.841949 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-var-lib-cni-multus\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.842593 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.842518 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-registration-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:01.842948 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:01.842617 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 12:51:01.842948 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.842620 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-sysctl-d\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.842948 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.842687 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-etc-selinux\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:01.842948 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.842875 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-device-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:01.842948 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.842911 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-slash\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.842948 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.842909 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/23d95081-1188-4002-997e-65ca57111fab-ovnkube-config\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.842948 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.842928 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-var-lib-cni-multus\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.843291 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:01.842981 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs podName:16a02457-151b-4e96-8118-e7695190b9be nodeName:}" failed. No retries permitted until 2026-10-30 12:51:02.342965364 +0000 UTC m=+3.065768316 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs") pod "network-metrics-daemon-24p2b" (UID: "16a02457-151b-4e96-8118-e7695190b9be") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 12:51:01.843291 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.843226 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:01.843291 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.843282 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/49b68250-16ee-42f1-9730-e4d326c96cd8-agent-certs\") pod \"konnectivity-agent-vjn55\" (UID: \"49b68250-16ee-42f1-9730-e4d326c96cd8\") " pod="kube-system/konnectivity-agent-vjn55" Oct 30 12:51:01.843448 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.843347 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/49b68250-16ee-42f1-9730-e4d326c96cd8-konnectivity-ca\") pod \"konnectivity-agent-vjn55\" (UID: \"49b68250-16ee-42f1-9730-e4d326c96cd8\") " pod="kube-system/konnectivity-agent-vjn55" Oct 30 12:51:01.843500 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.843425 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d-hosts-file\") pod \"node-resolver-759xk\" (UID: \"7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d\") " pod="openshift-dns/node-resolver-759xk" Oct 30 12:51:01.843549 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.843504 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-cni-netd\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.843549 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.843538 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/23d95081-1188-4002-997e-65ca57111fab-ovn-node-metrics-cert\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.843642 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.843570 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-lib-modules\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.843642 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.843610 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:01.843731 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.843644 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-var-lib-cni-bin\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.843731 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.843694 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2a12192e-f0c8-4354-b9fa-1b65b04ebc19-host\") pod \"node-ca-vsf9w\" (UID: \"2a12192e-f0c8-4354-b9fa-1b65b04ebc19\") " pod="openshift-image-registry/node-ca-vsf9w" Oct 30 12:51:01.843824 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.843728 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-log-socket\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.843824 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.843733 2583 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 12:51:01.843824 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.843763 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/23d95081-1188-4002-997e-65ca57111fab-env-overrides\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.843955 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.843902 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-systemd\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.843955 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.843945 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-run-netns\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.844049 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.843975 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-var-lib-openvswitch\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.844049 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844007 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/23d95081-1188-4002-997e-65ca57111fab-ovnkube-script-lib\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.844160 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844037 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-var-lib-kubelet\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.844160 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844102 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmqmx\" (UniqueName: \"kubernetes.io/projected/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-kube-api-access-pmqmx\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.844257 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844162 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9446374-a7ea-4f95-baa4-d6dad6805b65-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:01.844257 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844200 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd65z\" (UniqueName: \"kubernetes.io/projected/e9446374-a7ea-4f95-baa4-d6dad6805b65-kube-api-access-wd65z\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:01.844257 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844220 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-log-socket\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.844257 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844232 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-hostroot\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.844435 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844296 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vkl7\" (UniqueName: \"kubernetes.io/projected/3162c3d2-e86f-4ae1-8196-0414509fae9e-kube-api-access-7vkl7\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.844435 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844330 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/49b68250-16ee-42f1-9730-e4d326c96cd8-konnectivity-ca\") pod \"konnectivity-agent-vjn55\" (UID: \"49b68250-16ee-42f1-9730-e4d326c96cd8\") " pod="kube-system/konnectivity-agent-vjn55" Oct 30 12:51:01.844435 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844356 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d6a61ca-a15f-4742-8e81-6057c23e51c8-iptables-alerter-script\") pod \"iptables-alerter-2cvn5\" (UID: \"0d6a61ca-a15f-4742-8e81-6057c23e51c8\") " pod="openshift-network-operator/iptables-alerter-2cvn5" Oct 30 12:51:01.844435 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844418 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-modprobe-d\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.844665 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844459 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-host\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.844665 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844540 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-tuned\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.844665 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844574 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-tmp\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.844665 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844601 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9446374-a7ea-4f95-baa4-d6dad6805b65-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:01.844665 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844634 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-kubelet\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.844897 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844667 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-systemd-units\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.844897 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844713 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-run-ovn\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.844897 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844775 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-kubernetes\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.844897 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844819 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mld9s\" (UniqueName: \"kubernetes.io/projected/2a12192e-f0c8-4354-b9fa-1b65b04ebc19-kube-api-access-mld9s\") pod \"node-ca-vsf9w\" (UID: \"2a12192e-f0c8-4354-b9fa-1b65b04ebc19\") " pod="openshift-image-registry/node-ca-vsf9w" Oct 30 12:51:01.844897 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844849 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-run-systemd\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.845130 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844906 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-sysconfig\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.845130 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844936 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-sysctl-conf\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.845130 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.844976 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-socket-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:01.845130 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.845007 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-sys-fs\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:01.845130 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.845030 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d6a61ca-a15f-4742-8e81-6057c23e51c8-iptables-alerter-script\") pod \"iptables-alerter-2cvn5\" (UID: \"0d6a61ca-a15f-4742-8e81-6057c23e51c8\") " pod="openshift-network-operator/iptables-alerter-2cvn5" Oct 30 12:51:01.845130 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.845074 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-run-k8s-cni-cncf-io\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.845424 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.845200 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-systemd\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.845424 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.845256 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc5nm\" (UniqueName: \"kubernetes.io/projected/16a02457-151b-4e96-8118-e7695190b9be-kube-api-access-dc5nm\") pod \"network-metrics-daemon-24p2b\" (UID: \"16a02457-151b-4e96-8118-e7695190b9be\") " pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:01.845424 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.845294 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-run-netns\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.845424 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.845371 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-var-lib-openvswitch\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.845424 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.845396 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-run\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.845661 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.845427 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9446374-a7ea-4f95-baa4-d6dad6805b65-cnibin\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:01.845661 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.845459 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-system-cni-dir\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.845661 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.845496 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-cnibin\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.845661 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.845526 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-daemon-config\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.845661 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.845554 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-etc-openvswitch\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.845661 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.845594 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-node-log\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.845918 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.845652 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9446374-a7ea-4f95-baa4-d6dad6805b65-system-cni-dir\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:01.845918 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.845705 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9446374-a7ea-4f95-baa4-d6dad6805b65-cni-binary-copy\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:01.845918 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.845737 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9446374-a7ea-4f95-baa4-d6dad6805b65-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:01.845918 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.845805 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-conf-dir\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.846056 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.845920 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2a12192e-f0c8-4354-b9fa-1b65b04ebc19-host\") pod \"node-ca-vsf9w\" (UID: \"2a12192e-f0c8-4354-b9fa-1b65b04ebc19\") " pod="openshift-image-registry/node-ca-vsf9w" Oct 30 12:51:01.846056 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.845969 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-cni-netd\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.846056 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.845977 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-socket-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:01.846056 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.845984 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/23d95081-1188-4002-997e-65ca57111fab-ovnkube-script-lib\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.846258 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.846100 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:01.846317 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:01.846244 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 12:51:01.846376 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.846337 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-var-lib-kubelet\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.846376 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.846344 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-var-lib-cni-bin\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.846376 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.846344 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-lib-modules\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.846602 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.846579 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-hostroot\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.846671 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.846628 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d-hosts-file\") pod \"node-resolver-759xk\" (UID: \"7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d\") " pod="openshift-dns/node-resolver-759xk" Oct 30 12:51:01.846797 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.846777 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-modprobe-d\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.846859 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.846817 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-systemd-units\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.846916 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.846861 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-kubernetes\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.846916 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.846893 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-kubelet\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.847021 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.846940 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/23d95081-1188-4002-997e-65ca57111fab-env-overrides\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.847021 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.846978 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-run-ovn\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.847021 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:01.846982 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret podName:75bd9ac5-49ff-4c58-85c7-79b181150555 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:02.346963864 +0000 UTC m=+3.069766833 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret") pod "global-pull-secret-syncer-lf7b2" (UID: "75bd9ac5-49ff-4c58-85c7-79b181150555") : object "kube-system"/"original-pull-secret" not registered Oct 30 12:51:01.847207 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.847030 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-host\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.847207 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.847036 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-cnibin\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.847639 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.847532 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-sysconfig\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.847639 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.847623 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9446374-a7ea-4f95-baa4-d6dad6805b65-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:01.847820 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.847792 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9446374-a7ea-4f95-baa4-d6dad6805b65-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:01.847881 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.847841 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-sysctl-conf\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.847936 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.847922 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-run-systemd\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.848079 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.848046 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-sys-fs\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:01.848158 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.848082 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-conf-dir\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.848158 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.848139 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-run-k8s-cni-cncf-io\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.848270 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.848251 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-system-cni-dir\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.848324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.848257 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9446374-a7ea-4f95-baa4-d6dad6805b65-cnibin\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:01.848324 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.848313 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-run\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.849185 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.848482 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9446374-a7ea-4f95-baa4-d6dad6805b65-system-cni-dir\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:01.849185 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.848442 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-node-log\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.849185 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.848505 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-etc-openvswitch\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.849185 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.848601 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9446374-a7ea-4f95-baa4-d6dad6805b65-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:01.849185 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.848691 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9446374-a7ea-4f95-baa4-d6dad6805b65-cni-binary-copy\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:01.849185 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.849091 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-daemon-config\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.850057 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.850033 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/23d95081-1188-4002-997e-65ca57111fab-ovn-node-metrics-cert\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.851128 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.851052 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/49b68250-16ee-42f1-9730-e4d326c96cd8-agent-certs\") pod \"konnectivity-agent-vjn55\" (UID: \"49b68250-16ee-42f1-9730-e4d326c96cd8\") " pod="kube-system/konnectivity-agent-vjn55" Oct 30 12:51:01.851638 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.851612 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-tuned\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.852769 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.852747 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-tmp\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.871031 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.871010 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8hrc\" (UniqueName: \"kubernetes.io/projected/23d95081-1188-4002-997e-65ca57111fab-kube-api-access-j8hrc\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:01.871530 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.871497 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvwvc\" (UniqueName: \"kubernetes.io/projected/7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d-kube-api-access-mvwvc\") pod \"node-resolver-759xk\" (UID: \"7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d\") " pod="openshift-dns/node-resolver-759xk" Oct 30 12:51:01.874684 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:01.874664 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 12:51:01.874769 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:01.874691 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 12:51:01.874769 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:01.874728 2583 projected.go:196] Error preparing data for projected volume kube-api-access-q6zpc for pod openshift-network-diagnostics/network-check-target-d6rt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 12:51:01.874905 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:01.874790 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc podName:a5428d1a-8c8b-41ce-ad97-a729ec7a2e43 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:02.374772314 +0000 UTC m=+3.097575279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-q6zpc" (UniqueName: "kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc") pod "network-check-target-d6rt6" (UID: "a5428d1a-8c8b-41ce-ad97-a729ec7a2e43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 12:51:01.877532 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.877511 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5djfp\" (UniqueName: \"kubernetes.io/projected/0d6a61ca-a15f-4742-8e81-6057c23e51c8-kube-api-access-5djfp\") pod \"iptables-alerter-2cvn5\" (UID: \"0d6a61ca-a15f-4742-8e81-6057c23e51c8\") " pod="openshift-network-operator/iptables-alerter-2cvn5" Oct 30 12:51:01.879371 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.879350 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-997qf\" (UniqueName: \"kubernetes.io/projected/0b1634e9-3028-4b6a-b651-9155018b3c5a-kube-api-access-997qf\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:01.884029 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.884005 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mld9s\" (UniqueName: \"kubernetes.io/projected/2a12192e-f0c8-4354-b9fa-1b65b04ebc19-kube-api-access-mld9s\") pod \"node-ca-vsf9w\" (UID: \"2a12192e-f0c8-4354-b9fa-1b65b04ebc19\") " pod="openshift-image-registry/node-ca-vsf9w" Oct 30 12:51:01.884687 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.884659 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd65z\" (UniqueName: \"kubernetes.io/projected/e9446374-a7ea-4f95-baa4-d6dad6805b65-kube-api-access-wd65z\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:01.885300 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.885251 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vkl7\" (UniqueName: \"kubernetes.io/projected/3162c3d2-e86f-4ae1-8196-0414509fae9e-kube-api-access-7vkl7\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:01.890097 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.890048 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmqmx\" (UniqueName: \"kubernetes.io/projected/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-kube-api-access-pmqmx\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:01.890877 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.890858 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc5nm\" (UniqueName: \"kubernetes.io/projected/16a02457-151b-4e96-8118-e7695190b9be-kube-api-access-dc5nm\") pod \"network-metrics-daemon-24p2b\" (UID: \"16a02457-151b-4e96-8118-e7695190b9be\") " pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:01.972212 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.972183 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vjn55" Oct 30 12:51:01.982194 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:51:01.982164 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49b68250_16ee_42f1_9730_e4d326c96cd8.slice/crio-5d058b5a3ab026a3bb963d88805fdecb2c35b5a44bd7181f921f4a1ac848f65f WatchSource:0}: Error finding container 5d058b5a3ab026a3bb963d88805fdecb2c35b5a44bd7181f921f4a1ac848f65f: Status 404 returned error can't find the container with id 5d058b5a3ab026a3bb963d88805fdecb2c35b5a44bd7181f921f4a1ac848f65f Oct 30 12:51:01.984208 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:01.984019 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2cvn5" Oct 30 12:51:01.997695 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:51:01.997666 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d6a61ca_a15f_4742_8e81_6057c23e51c8.slice/crio-f266024b642e8c0a51cea8631c67704b363aa8bd6ba2b134d581bf3e42ccfc5a WatchSource:0}: Error finding container f266024b642e8c0a51cea8631c67704b363aa8bd6ba2b134d581bf3e42ccfc5a: Status 404 returned error can't find the container with id f266024b642e8c0a51cea8631c67704b363aa8bd6ba2b134d581bf3e42ccfc5a Oct 30 12:51:02.002837 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.002802 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:02.002946 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.002913 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:02.009532 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.009513 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:02.012081 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:51:02.012041 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23d95081_1188_4002_997e_65ca57111fab.slice/crio-24302e1400dcaebd4cb16e81d45385e0c915c341c9816031cf4b7e45c2174779 WatchSource:0}: Error finding container 24302e1400dcaebd4cb16e81d45385e0c915c341c9816031cf4b7e45c2174779: Status 404 returned error can't find the container with id 24302e1400dcaebd4cb16e81d45385e0c915c341c9816031cf4b7e45c2174779 Oct 30 12:51:02.014894 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.014875 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vsf9w" Oct 30 12:51:02.016634 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:51:02.016593 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4c9c543_f712_4a48_949f_d3f3bc8f77e1.slice/crio-b8b7214fc873c4c3f498bac37c9c64e847b194b237a13749cfc3b348a7effd7d WatchSource:0}: Error finding container b8b7214fc873c4c3f498bac37c9c64e847b194b237a13749cfc3b348a7effd7d: Status 404 returned error can't find the container with id b8b7214fc873c4c3f498bac37c9c64e847b194b237a13749cfc3b348a7effd7d Oct 30 12:51:02.019971 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:51:02.019944 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b1634e9_3028_4b6a_b651_9155018b3c5a.slice/crio-293b3487250d7ffce1669a266424a8db29e6a9d7bf5d24e4d36e5c0fecd1ef44 WatchSource:0}: Error finding container 293b3487250d7ffce1669a266424a8db29e6a9d7bf5d24e4d36e5c0fecd1ef44: Status 404 returned error can't find the container with id 293b3487250d7ffce1669a266424a8db29e6a9d7bf5d24e4d36e5c0fecd1ef44 Oct 30 12:51:02.023466 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.023304 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-759xk" Oct 30 12:51:02.033629 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:51:02.033596 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fdf07a3_cf6b_4fa7_9b90_6e92257d5d0d.slice/crio-c44099040f0f9dbde74f7f75f936ba3c25726e5142e589d606fb6828685e3a2f WatchSource:0}: Error finding container c44099040f0f9dbde74f7f75f936ba3c25726e5142e589d606fb6828685e3a2f: Status 404 returned error can't find the container with id c44099040f0f9dbde74f7f75f936ba3c25726e5142e589d606fb6828685e3a2f Oct 30 12:51:02.038976 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.038952 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:02.040151 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.040132 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hjpbf" Oct 30 12:51:02.054335 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:51:02.054299 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3162c3d2_e86f_4ae1_8196_0414509fae9e.slice/crio-42e3b567f84d63466cb877bf58cc08c2897a625926e6e52a4230fc9efeafa857 WatchSource:0}: Error finding container 42e3b567f84d63466cb877bf58cc08c2897a625926e6e52a4230fc9efeafa857: Status 404 returned error can't find the container with id 42e3b567f84d63466cb877bf58cc08c2897a625926e6e52a4230fc9efeafa857 Oct 30 12:51:02.055138 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:51:02.055101 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9446374_a7ea_4f95_baa4_d6dad6805b65.slice/crio-c84db34261e5eaa6a7c9f6a75c2bc81bfc62f530f40cf3ac52f021f126ddd617 WatchSource:0}: Error finding container c84db34261e5eaa6a7c9f6a75c2bc81bfc62f530f40cf3ac52f021f126ddd617: Status 404 returned error can't find the container with id c84db34261e5eaa6a7c9f6a75c2bc81bfc62f530f40cf3ac52f021f126ddd617 Oct 30 12:51:02.350292 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.350259 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs\") pod \"network-metrics-daemon-24p2b\" (UID: \"16a02457-151b-4e96-8118-e7695190b9be\") " pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:02.350451 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.350306 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:02.350451 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:02.350406 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 12:51:02.350569 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:02.350465 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs podName:16a02457-151b-4e96-8118-e7695190b9be nodeName:}" failed. No retries permitted until 2026-10-30 12:51:03.35045213 +0000 UTC m=+4.073255079 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs") pod "network-metrics-daemon-24p2b" (UID: "16a02457-151b-4e96-8118-e7695190b9be") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 12:51:02.350766 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:02.350406 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 12:51:02.350846 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:02.350812 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret podName:75bd9ac5-49ff-4c58-85c7-79b181150555 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:03.350793509 +0000 UTC m=+4.073596476 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret") pod "global-pull-secret-syncer-lf7b2" (UID: "75bd9ac5-49ff-4c58-85c7-79b181150555") : object "kube-system"/"original-pull-secret" not registered Oct 30 12:51:02.452406 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.451497 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6zpc\" (UniqueName: \"kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc\") pod \"network-check-target-d6rt6\" (UID: \"a5428d1a-8c8b-41ce-ad97-a729ec7a2e43\") " pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:02.452406 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:02.451923 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 12:51:02.452406 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:02.451940 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 12:51:02.452406 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:02.451950 2583 projected.go:196] Error preparing data for projected volume kube-api-access-q6zpc for pod openshift-network-diagnostics/network-check-target-d6rt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 12:51:02.452406 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:02.451992 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc podName:a5428d1a-8c8b-41ce-ad97-a729ec7a2e43 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:03.45197825 +0000 UTC m=+4.174781198 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-q6zpc" (UniqueName: "kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc") pod "network-check-target-d6rt6" (UID: "a5428d1a-8c8b-41ce-ad97-a729ec7a2e43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 12:51:02.731640 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.731480 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 12:46:02 +0000 UTC" deadline="2028-07-18 20:48:42.895069398 +0000 UTC" Oct 30 12:51:02.731640 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.731555 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15055h57m40.163519674s" Oct 30 12:51:02.779552 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.779516 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:02.779727 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:02.779653 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24p2b" podUID="16a02457-151b-4e96-8118-e7695190b9be" Oct 30 12:51:02.809888 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.809806 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s2xpm" event={"ID":"e9446374-a7ea-4f95-baa4-d6dad6805b65","Type":"ContainerStarted","Data":"c84db34261e5eaa6a7c9f6a75c2bc81bfc62f530f40cf3ac52f021f126ddd617"} Oct 30 12:51:02.815918 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.815831 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-759xk" event={"ID":"7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d","Type":"ContainerStarted","Data":"c44099040f0f9dbde74f7f75f936ba3c25726e5142e589d606fb6828685e3a2f"} Oct 30 12:51:02.819610 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.819583 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vsf9w" event={"ID":"2a12192e-f0c8-4354-b9fa-1b65b04ebc19","Type":"ContainerStarted","Data":"683b24075513d15c397f91afaf5097736db18b34fbf3597da3ac3094eac8cc95"} Oct 30 12:51:02.826757 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.826724 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" event={"ID":"23d95081-1188-4002-997e-65ca57111fab","Type":"ContainerStarted","Data":"24302e1400dcaebd4cb16e81d45385e0c915c341c9816031cf4b7e45c2174779"} Oct 30 12:51:02.841927 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.841892 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vjn55" event={"ID":"49b68250-16ee-42f1-9730-e4d326c96cd8","Type":"ContainerStarted","Data":"5d058b5a3ab026a3bb963d88805fdecb2c35b5a44bd7181f921f4a1ac848f65f"} Oct 30 12:51:02.864906 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.864094 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-147.ec2.internal" event={"ID":"7d0024b3080a170827a29027ec64e58f","Type":"ContainerStarted","Data":"8f91a6e7c2cea557ea711ab965883c40d084b0a5088b76cbcd22b035fc618fde"} Oct 30 12:51:02.873101 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.872431 2583 generic.go:356] "Generic (PLEG): container finished" podID="3f5d6256264944b543b87edf9d8fc228" containerID="655bad5f6df15aa40346ac1df3071fd2e74d1b2b9661432c9fc3e387afe23664" exitCode=0 Oct 30 12:51:02.873101 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.872522 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" event={"ID":"3f5d6256264944b543b87edf9d8fc228","Type":"ContainerDied","Data":"655bad5f6df15aa40346ac1df3071fd2e74d1b2b9661432c9fc3e387afe23664"} Oct 30 12:51:02.879920 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.879849 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hjpbf" event={"ID":"3162c3d2-e86f-4ae1-8196-0414509fae9e","Type":"ContainerStarted","Data":"42e3b567f84d63466cb877bf58cc08c2897a625926e6e52a4230fc9efeafa857"} Oct 30 12:51:02.889690 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.889658 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" event={"ID":"0b1634e9-3028-4b6a-b651-9155018b3c5a","Type":"ContainerStarted","Data":"293b3487250d7ffce1669a266424a8db29e6a9d7bf5d24e4d36e5c0fecd1ef44"} Oct 30 12:51:02.897952 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.897881 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" event={"ID":"c4c9c543-f712-4a48-949f-d3f3bc8f77e1","Type":"ContainerStarted","Data":"b8b7214fc873c4c3f498bac37c9c64e847b194b237a13749cfc3b348a7effd7d"} Oct 30 12:51:02.912810 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.912761 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-147.ec2.internal" podStartSLOduration=1.912743763 podStartE2EDuration="1.912743763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:51:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:02.912360932 +0000 UTC m=+3.635163909" watchObservedRunningTime="2026-10-30 12:51:02.912743763 +0000 UTC m=+3.635546736" Oct 30 12:51:02.926358 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:02.926289 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2cvn5" event={"ID":"0d6a61ca-a15f-4742-8e81-6057c23e51c8","Type":"ContainerStarted","Data":"f266024b642e8c0a51cea8631c67704b363aa8bd6ba2b134d581bf3e42ccfc5a"} Oct 30 12:51:03.360693 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:03.359810 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs\") pod \"network-metrics-daemon-24p2b\" (UID: \"16a02457-151b-4e96-8118-e7695190b9be\") " pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:03.360693 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:03.359879 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:03.360693 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:03.360039 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 12:51:03.360693 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:03.360104 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret podName:75bd9ac5-49ff-4c58-85c7-79b181150555 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:05.360084597 +0000 UTC m=+6.082887564 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret") pod "global-pull-secret-syncer-lf7b2" (UID: "75bd9ac5-49ff-4c58-85c7-79b181150555") : object "kube-system"/"original-pull-secret" not registered Oct 30 12:51:03.360693 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:03.360226 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 12:51:03.360693 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:03.360268 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs podName:16a02457-151b-4e96-8118-e7695190b9be nodeName:}" failed. No retries permitted until 2026-10-30 12:51:05.36025427 +0000 UTC m=+6.083057233 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs") pod "network-metrics-daemon-24p2b" (UID: "16a02457-151b-4e96-8118-e7695190b9be") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 12:51:03.461097 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:03.460437 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6zpc\" (UniqueName: \"kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc\") pod \"network-check-target-d6rt6\" (UID: \"a5428d1a-8c8b-41ce-ad97-a729ec7a2e43\") " pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:03.461097 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:03.460653 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 12:51:03.461097 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:03.460674 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 12:51:03.461097 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:03.460686 2583 projected.go:196] Error preparing data for projected volume kube-api-access-q6zpc for pod openshift-network-diagnostics/network-check-target-d6rt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 12:51:03.461097 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:03.460748 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc podName:a5428d1a-8c8b-41ce-ad97-a729ec7a2e43 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:05.460727551 +0000 UTC m=+6.183530520 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-q6zpc" (UniqueName: "kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc") pod "network-check-target-d6rt6" (UID: "a5428d1a-8c8b-41ce-ad97-a729ec7a2e43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 12:51:03.732667 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:03.732572 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 12:46:02 +0000 UTC" deadline="2028-04-07 22:04:59.370117904 +0000 UTC" Oct 30 12:51:03.732667 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:03.732616 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12609h13m55.637507262s" Oct 30 12:51:03.786446 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:03.781978 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:03.786446 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:03.782117 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6rt6" podUID="a5428d1a-8c8b-41ce-ad97-a729ec7a2e43" Oct 30 12:51:03.786446 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:03.782225 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:03.786446 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:03.782301 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lf7b2" podUID="75bd9ac5-49ff-4c58-85c7-79b181150555" Oct 30 12:51:03.943245 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:03.943213 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" event={"ID":"3f5d6256264944b543b87edf9d8fc228","Type":"ContainerStarted","Data":"134f2a06d7e58292517228c2edb72b425436a1c722c3322122e6be671d1717b7"} Oct 30 12:51:04.779149 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:04.778632 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:04.779149 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:04.778775 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24p2b" podUID="16a02457-151b-4e96-8118-e7695190b9be" Oct 30 12:51:04.948344 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:04.948304 2583 generic.go:356] "Generic (PLEG): container finished" podID="e9446374-a7ea-4f95-baa4-d6dad6805b65" containerID="52a11e7920fe35baa1e8429608f4594144aaad9e47e07ab210c67fc3131b470e" exitCode=0 Oct 30 12:51:04.949464 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:04.949438 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s2xpm" event={"ID":"e9446374-a7ea-4f95-baa4-d6dad6805b65","Type":"ContainerDied","Data":"52a11e7920fe35baa1e8429608f4594144aaad9e47e07ab210c67fc3131b470e"} Oct 30 12:51:05.037164 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:05.036690 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" podStartSLOduration=4.036670656 podStartE2EDuration="4.036670656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:51:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:03.991774486 +0000 UTC m=+4.714577454" watchObservedRunningTime="2026-10-30 12:51:05.036670656 +0000 UTC m=+5.759473629" Oct 30 12:51:05.376823 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:05.376735 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs\") pod \"network-metrics-daemon-24p2b\" (UID: \"16a02457-151b-4e96-8118-e7695190b9be\") " pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:05.376823 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:05.376799 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:05.377038 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:05.376934 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 12:51:05.377038 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:05.376995 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret podName:75bd9ac5-49ff-4c58-85c7-79b181150555 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:09.376973786 +0000 UTC m=+10.099776738 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret") pod "global-pull-secret-syncer-lf7b2" (UID: "75bd9ac5-49ff-4c58-85c7-79b181150555") : object "kube-system"/"original-pull-secret" not registered Oct 30 12:51:05.377491 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:05.377420 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 12:51:05.377491 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:05.377477 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs podName:16a02457-151b-4e96-8118-e7695190b9be nodeName:}" failed. No retries permitted until 2026-10-30 12:51:09.377461943 +0000 UTC m=+10.100264895 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs") pod "network-metrics-daemon-24p2b" (UID: "16a02457-151b-4e96-8118-e7695190b9be") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 12:51:05.477418 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:05.477383 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6zpc\" (UniqueName: \"kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc\") pod \"network-check-target-d6rt6\" (UID: \"a5428d1a-8c8b-41ce-ad97-a729ec7a2e43\") " pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:05.477583 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:05.477571 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 12:51:05.477658 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:05.477590 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 12:51:05.477658 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:05.477606 2583 projected.go:196] Error preparing data for projected volume kube-api-access-q6zpc for pod openshift-network-diagnostics/network-check-target-d6rt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 12:51:05.477773 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:05.477660 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc podName:a5428d1a-8c8b-41ce-ad97-a729ec7a2e43 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:09.477642254 +0000 UTC m=+10.200445204 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-q6zpc" (UniqueName: "kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc") pod "network-check-target-d6rt6" (UID: "a5428d1a-8c8b-41ce-ad97-a729ec7a2e43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 12:51:05.780982 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:05.780895 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:05.780982 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:05.780937 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:05.781523 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:05.781099 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lf7b2" podUID="75bd9ac5-49ff-4c58-85c7-79b181150555" Oct 30 12:51:05.781683 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:05.781657 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6rt6" podUID="a5428d1a-8c8b-41ce-ad97-a729ec7a2e43" Oct 30 12:51:06.783904 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:06.783358 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:06.783904 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:06.783517 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24p2b" podUID="16a02457-151b-4e96-8118-e7695190b9be" Oct 30 12:51:06.955972 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:06.955927 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vjn55" event={"ID":"49b68250-16ee-42f1-9730-e4d326c96cd8","Type":"ContainerStarted","Data":"07616f58590fe2030be264a5f399c742be49a340ef3f15273c2483d3291437ec"} Oct 30 12:51:07.009263 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:07.009162 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vjn55" podStartSLOduration=3.099245307 podStartE2EDuration="7.009128212s" totalImagesPullingTime="3.909882905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:51:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:07.007975438 +0000 UTC m=+7.730778414" watchObservedRunningTime="2026-10-30 12:51:07.009128212 +0000 UTC m=+7.731931184" Oct 30 12:51:07.779248 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:07.779215 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:07.779445 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:07.779354 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lf7b2" podUID="75bd9ac5-49ff-4c58-85c7-79b181150555" Oct 30 12:51:07.781277 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:07.781241 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:07.781396 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:07.781364 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6rt6" podUID="a5428d1a-8c8b-41ce-ad97-a729ec7a2e43" Oct 30 12:51:07.786843 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:07.786814 2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vjn55" Oct 30 12:51:07.968582 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:07.968542 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vsf9w" event={"ID":"2a12192e-f0c8-4354-b9fa-1b65b04ebc19","Type":"ContainerStarted","Data":"0869b9d5c290bcb5eee5944deca03764c9cb2f207e562ad953ed1e1856058041"} Oct 30 12:51:08.016646 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:08.015927 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vsf9w" podStartSLOduration=2.854502421 podStartE2EDuration="8.015906627s" totalImagesPullingTime="5.161404206s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:51:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:08.015659432 +0000 UTC m=+8.738462404" watchObservedRunningTime="2026-10-30 12:51:08.015906627 +0000 UTC m=+8.738709604" Oct 30 12:51:08.779256 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:08.779182 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:08.779389 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:08.779334 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24p2b" podUID="16a02457-151b-4e96-8118-e7695190b9be" Oct 30 12:51:08.973623 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:08.973588 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" event={"ID":"0b1634e9-3028-4b6a-b651-9155018b3c5a","Type":"ContainerStarted","Data":"3e6be1956902ec1d46c375414b86d82a9ea76129ce6241bdbb22cca6d4605c36"} Oct 30 12:51:09.414677 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:09.414254 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:09.414677 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:09.414336 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs\") pod \"network-metrics-daemon-24p2b\" (UID: \"16a02457-151b-4e96-8118-e7695190b9be\") " pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:09.414677 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:09.414463 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 12:51:09.414677 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:09.414518 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs podName:16a02457-151b-4e96-8118-e7695190b9be nodeName:}" failed. No retries permitted until 2026-10-30 12:51:17.414499122 +0000 UTC m=+18.137302085 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs") pod "network-metrics-daemon-24p2b" (UID: "16a02457-151b-4e96-8118-e7695190b9be") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 12:51:09.414677 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:09.414579 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 12:51:09.414677 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:09.414612 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret podName:75bd9ac5-49ff-4c58-85c7-79b181150555 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:17.414600589 +0000 UTC m=+18.137403570 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret") pod "global-pull-secret-syncer-lf7b2" (UID: "75bd9ac5-49ff-4c58-85c7-79b181150555") : object "kube-system"/"original-pull-secret" not registered Oct 30 12:51:09.516233 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:09.515657 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6zpc\" (UniqueName: \"kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc\") pod \"network-check-target-d6rt6\" (UID: \"a5428d1a-8c8b-41ce-ad97-a729ec7a2e43\") " pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:09.516233 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:09.515867 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 12:51:09.516233 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:09.515894 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 12:51:09.516233 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:09.515908 2583 projected.go:196] Error preparing data for projected volume kube-api-access-q6zpc for pod openshift-network-diagnostics/network-check-target-d6rt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 12:51:09.516233 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:09.515949 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc podName:a5428d1a-8c8b-41ce-ad97-a729ec7a2e43 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:17.515936629 +0000 UTC m=+18.238739589 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-q6zpc" (UniqueName: "kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc") pod "network-check-target-d6rt6" (UID: "a5428d1a-8c8b-41ce-ad97-a729ec7a2e43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 12:51:09.599720 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:09.598807 2583 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 12:51:09.696273 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:09.696125 2583 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T12:51:09.598828225Z","UUID":"616dec21-3c74-4b08-b3eb-55f3916a8850","Handler":null,"Name":"","Endpoint":""} Oct 30 12:51:09.705998 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:09.705318 2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 12:51:09.705998 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:09.705363 2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 12:51:09.781094 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:09.778529 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:09.781094 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:09.778655 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6rt6" podUID="a5428d1a-8c8b-41ce-ad97-a729ec7a2e43" Oct 30 12:51:09.781094 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:09.779002 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:09.781094 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:09.779123 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lf7b2" podUID="75bd9ac5-49ff-4c58-85c7-79b181150555" Oct 30 12:51:09.988623 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:09.988463 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" event={"ID":"0b1634e9-3028-4b6a-b651-9155018b3c5a","Type":"ContainerStarted","Data":"5069772af1affdc4781393cc6587121dc4f443edf11938e3d08f6047ffb62b88"} Oct 30 12:51:10.471804 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:10.468059 2583 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vjn55" Oct 30 12:51:10.471804 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:10.468662 2583 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vjn55" Oct 30 12:51:10.779181 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:10.779150 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:10.779325 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:10.779282 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24p2b" podUID="16a02457-151b-4e96-8118-e7695190b9be" Oct 30 12:51:10.994186 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:10.993367 2583 generic.go:356] "Generic (PLEG): container finished" podID="e9446374-a7ea-4f95-baa4-d6dad6805b65" containerID="846cde979c42d7b1e3ca8a1d674e0dacb10ed1ae6d8cd660ca4e2ff337e16b73" exitCode=0 Oct 30 12:51:10.994186 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:10.993456 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s2xpm" event={"ID":"e9446374-a7ea-4f95-baa4-d6dad6805b65","Type":"ContainerDied","Data":"846cde979c42d7b1e3ca8a1d674e0dacb10ed1ae6d8cd660ca4e2ff337e16b73"} Oct 30 12:51:10.995953 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:10.995918 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-759xk" event={"ID":"7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d","Type":"ContainerStarted","Data":"02abc126aead6dfea35d1e86f6b700a0fc517d452b174242c3133009f9a98022"} Oct 30 12:51:10.997658 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:10.997633 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hjpbf" event={"ID":"3162c3d2-e86f-4ae1-8196-0414509fae9e","Type":"ContainerStarted","Data":"209fc37a4af488fddd04588d10c9b8ab1b382d8fd7dcb744a0a478b5ce03d246"} Oct 30 12:51:11.001187 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:11.001017 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" event={"ID":"0b1634e9-3028-4b6a-b651-9155018b3c5a","Type":"ContainerStarted","Data":"d64f701d68f35474fd9bdcbc760f6ca7947528717d48044410294fa5acdc085b"} Oct 30 12:51:11.001501 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:11.001482 2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vjn55" Oct 30 12:51:11.140599 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:11.140550 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-hjpbf" podStartSLOduration=3.222406403 podStartE2EDuration="11.1405362s" totalImagesPullingTime="7.918129797s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:51:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:11.140343792 +0000 UTC m=+11.863146761" watchObservedRunningTime="2026-10-30 12:51:11.1405362 +0000 UTC m=+11.863339167" Oct 30 12:51:11.188534 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:11.188478 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" podStartSLOduration=3.09463037 podStartE2EDuration="11.18845831s" totalImagesPullingTime="8.09382794s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:51:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:11.187717063 +0000 UTC m=+11.910520032" watchObservedRunningTime="2026-10-30 12:51:11.18845831 +0000 UTC m=+11.911261280" Oct 30 12:51:11.778616 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:11.778592 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:11.778720 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:11.778615 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:11.778778 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:11.778716 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6rt6" podUID="a5428d1a-8c8b-41ce-ad97-a729ec7a2e43" Oct 30 12:51:11.778828 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:11.778809 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lf7b2" podUID="75bd9ac5-49ff-4c58-85c7-79b181150555" Oct 30 12:51:12.011550 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:12.011513 2583 generic.go:356] "Generic (PLEG): container finished" podID="e9446374-a7ea-4f95-baa4-d6dad6805b65" containerID="0bb3feb1bc32a7dc61595f07d20cef372de3574fff34c2cbdf0be697fb45c4f7" exitCode=0 Oct 30 12:51:12.012137 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:12.011607 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s2xpm" event={"ID":"e9446374-a7ea-4f95-baa4-d6dad6805b65","Type":"ContainerDied","Data":"0bb3feb1bc32a7dc61595f07d20cef372de3574fff34c2cbdf0be697fb45c4f7"} Oct 30 12:51:12.068413 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:12.068359 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-759xk" podStartSLOduration=3.75050291 podStartE2EDuration="12.068341571s" totalImagesPullingTime="8.317838661s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:51:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:11.236058859 +0000 UTC m=+11.958861829" watchObservedRunningTime="2026-10-30 12:51:12.068341571 +0000 UTC m=+12.791144540" Oct 30 12:51:12.779319 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:12.779290 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:12.779447 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:12.779404 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24p2b" podUID="16a02457-151b-4e96-8118-e7695190b9be" Oct 30 12:51:13.015752 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:13.015667 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" event={"ID":"c4c9c543-f712-4a48-949f-d3f3bc8f77e1","Type":"ContainerStarted","Data":"a327376e46c736302c84b90ae44bd0691ee7dc309f9ff7514bafea1cfb7ef5b2"} Oct 30 12:51:13.017489 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:13.017452 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2cvn5" event={"ID":"0d6a61ca-a15f-4742-8e81-6057c23e51c8","Type":"ContainerStarted","Data":"d8632536e0e4190fd6c83c55a3a1fef1257df8005b1333ec83fed18bb9ed554d"} Oct 30 12:51:13.020227 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:13.020201 2583 generic.go:356] "Generic (PLEG): container finished" podID="e9446374-a7ea-4f95-baa4-d6dad6805b65" containerID="3e594a5b0da345b7bbaf500a20195a2b48901c88e697feb6d5b61b3c97671274" exitCode=0 Oct 30 12:51:13.020304 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:13.020242 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s2xpm" event={"ID":"e9446374-a7ea-4f95-baa4-d6dad6805b65","Type":"ContainerDied","Data":"3e594a5b0da345b7bbaf500a20195a2b48901c88e697feb6d5b61b3c97671274"} Oct 30 12:51:13.065516 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:13.065463 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" podStartSLOduration=2.933978004 podStartE2EDuration="13.06545008s" totalImagesPullingTime="10.131472076s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:51:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:13.065169299 +0000 UTC m=+13.787972278" watchObservedRunningTime="2026-10-30 12:51:13.06545008 +0000 UTC m=+13.788253044" Oct 30 12:51:13.111648 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:13.111586 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2cvn5" podStartSLOduration=4.848259415 podStartE2EDuration="13.111573268s" totalImagesPullingTime="8.263313853s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:51:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:13.111055522 +0000 UTC m=+13.833858499" watchObservedRunningTime="2026-10-30 12:51:13.111573268 +0000 UTC m=+13.834376249" Oct 30 12:51:13.779140 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:13.779102 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:13.779140 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:13.779125 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:13.779364 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:13.779249 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6rt6" podUID="a5428d1a-8c8b-41ce-ad97-a729ec7a2e43" Oct 30 12:51:13.779454 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:13.779421 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lf7b2" podUID="75bd9ac5-49ff-4c58-85c7-79b181150555" Oct 30 12:51:14.779271 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:14.779238 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:14.779870 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:14.779374 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24p2b" podUID="16a02457-151b-4e96-8118-e7695190b9be" Oct 30 12:51:15.027278 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:15.027092 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s2xpm" event={"ID":"e9446374-a7ea-4f95-baa4-d6dad6805b65","Type":"ContainerStarted","Data":"a22a214c42f5c88e59e2e505914fc2acf44e9f215a6ab18235d109ec0a5ef446"} Oct 30 12:51:15.782182 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:15.782149 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:15.782182 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:15.782187 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:15.782674 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:15.782259 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lf7b2" podUID="75bd9ac5-49ff-4c58-85c7-79b181150555" Oct 30 12:51:15.782674 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:15.782340 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6rt6" podUID="a5428d1a-8c8b-41ce-ad97-a729ec7a2e43" Oct 30 12:51:16.032510 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:16.032431 2583 generic.go:356] "Generic (PLEG): container finished" podID="e9446374-a7ea-4f95-baa4-d6dad6805b65" containerID="a22a214c42f5c88e59e2e505914fc2acf44e9f215a6ab18235d109ec0a5ef446" exitCode=0 Oct 30 12:51:16.032510 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:16.032493 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s2xpm" event={"ID":"e9446374-a7ea-4f95-baa4-d6dad6805b65","Type":"ContainerDied","Data":"a22a214c42f5c88e59e2e505914fc2acf44e9f215a6ab18235d109ec0a5ef446"} Oct 30 12:51:16.779207 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:16.779182 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:16.779329 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:16.779307 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24p2b" podUID="16a02457-151b-4e96-8118-e7695190b9be" Oct 30 12:51:17.037957 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:17.037727 2583 generic.go:356] "Generic (PLEG): container finished" podID="e9446374-a7ea-4f95-baa4-d6dad6805b65" containerID="abd64f2cecd97ddaacd2d62288e00b695e39f913c11265a59a417c92088604dc" exitCode=0 Oct 30 12:51:17.038399 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:17.037761 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s2xpm" event={"ID":"e9446374-a7ea-4f95-baa4-d6dad6805b65","Type":"ContainerDied","Data":"abd64f2cecd97ddaacd2d62288e00b695e39f913c11265a59a417c92088604dc"} Oct 30 12:51:17.041625 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:17.041606 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" event={"ID":"23d95081-1188-4002-997e-65ca57111fab","Type":"ContainerStarted","Data":"0d143b324291c53f89e27aa186cfb731d9848473464e02cd195e267bb12895a5"} Oct 30 12:51:17.041748 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:17.041668 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" event={"ID":"23d95081-1188-4002-997e-65ca57111fab","Type":"ContainerStarted","Data":"bc1d90070bb9a8b7afcf7fff3c5741a342af76ff8298cf4129d7397cad6f23c3"} Oct 30 12:51:17.041748 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:17.041698 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" event={"ID":"23d95081-1188-4002-997e-65ca57111fab","Type":"ContainerStarted","Data":"7c4e5f5aad3f7b1994ff5a716fe2ed7e65ec1ca050558c1349333c9a5bf89b4f"} Oct 30 12:51:17.041748 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:17.041726 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" event={"ID":"23d95081-1188-4002-997e-65ca57111fab","Type":"ContainerStarted","Data":"ce4d4cd39b0c5030b77a007d9a3514f3a2e810abc72f636b8857882a8d0b453b"} Oct 30 12:51:17.041891 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:17.041751 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" event={"ID":"23d95081-1188-4002-997e-65ca57111fab","Type":"ContainerStarted","Data":"0eefc2f4a29af44caea985eaeddcbf6c4017d3adae0b42e31cc2990d0a2c31b6"} Oct 30 12:51:17.041891 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:17.041779 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" event={"ID":"23d95081-1188-4002-997e-65ca57111fab","Type":"ContainerStarted","Data":"a118f68a9334299190ab7b4143c64e3930c8dc4311da9f9242d3af37b00fec1c"} Oct 30 12:51:17.476340 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:17.476307 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs\") pod \"network-metrics-daemon-24p2b\" (UID: \"16a02457-151b-4e96-8118-e7695190b9be\") " pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:17.476499 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:17.476350 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:17.476499 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:17.476431 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 12:51:17.476499 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:17.476445 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 12:51:17.476499 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:17.476476 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret podName:75bd9ac5-49ff-4c58-85c7-79b181150555 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:33.476465216 +0000 UTC m=+34.199268176 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret") pod "global-pull-secret-syncer-lf7b2" (UID: "75bd9ac5-49ff-4c58-85c7-79b181150555") : object "kube-system"/"original-pull-secret" not registered Oct 30 12:51:17.476499 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:17.476498 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs podName:16a02457-151b-4e96-8118-e7695190b9be nodeName:}" failed. No retries permitted until 2026-10-30 12:51:33.476481596 +0000 UTC m=+34.199284556 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs") pod "network-metrics-daemon-24p2b" (UID: "16a02457-151b-4e96-8118-e7695190b9be") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 12:51:17.577607 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:17.577574 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6zpc\" (UniqueName: \"kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc\") pod \"network-check-target-d6rt6\" (UID: \"a5428d1a-8c8b-41ce-ad97-a729ec7a2e43\") " pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:17.577747 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:17.577698 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 12:51:17.577747 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:17.577711 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 12:51:17.577747 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:17.577720 2583 projected.go:196] Error preparing data for projected volume kube-api-access-q6zpc for pod openshift-network-diagnostics/network-check-target-d6rt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 12:51:17.577838 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:17.577769 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc podName:a5428d1a-8c8b-41ce-ad97-a729ec7a2e43 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:33.577755652 +0000 UTC m=+34.300558611 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-q6zpc" (UniqueName: "kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc") pod "network-check-target-d6rt6" (UID: "a5428d1a-8c8b-41ce-ad97-a729ec7a2e43") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 12:51:17.781566 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:17.781492 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:17.781566 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:17.781525 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:17.781747 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:17.781615 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lf7b2" podUID="75bd9ac5-49ff-4c58-85c7-79b181150555" Oct 30 12:51:17.781747 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:17.781727 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6rt6" podUID="a5428d1a-8c8b-41ce-ad97-a729ec7a2e43" Oct 30 12:51:18.046507 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:18.046426 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-s2xpm" event={"ID":"e9446374-a7ea-4f95-baa4-d6dad6805b65","Type":"ContainerStarted","Data":"c4b2b32cd77a7c47df3dbbf7eaf34d790414b6737240d2673f9843e6a71cf561"} Oct 30 12:51:18.128256 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:18.128209 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-s2xpm" podStartSLOduration=7.773824205 podStartE2EDuration="18.128194361s" totalImagesPullingTime="10.354370156s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:51:00 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:18.113939694 +0000 UTC m=+18.836742662" watchObservedRunningTime="2026-10-30 12:51:18.128194361 +0000 UTC m=+18.850997329" Oct 30 12:51:18.779346 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:18.779317 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:18.779482 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:18.779423 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24p2b" podUID="16a02457-151b-4e96-8118-e7695190b9be" Oct 30 12:51:19.052347 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:19.052287 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" event={"ID":"23d95081-1188-4002-997e-65ca57111fab","Type":"ContainerStarted","Data":"be95cdf1d6c208ae7a8c622c47227e9d819a20f16a18b5fab01e9a36c9dc81d7"} Oct 30 12:51:19.780017 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:19.779984 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:19.780167 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:19.780102 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lf7b2" podUID="75bd9ac5-49ff-4c58-85c7-79b181150555" Oct 30 12:51:19.780222 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:19.780166 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:19.780282 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:19.780258 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6rt6" podUID="a5428d1a-8c8b-41ce-ad97-a729ec7a2e43" Oct 30 12:51:20.779250 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:20.779224 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:20.779706 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:20.779323 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24p2b" podUID="16a02457-151b-4e96-8118-e7695190b9be" Oct 30 12:51:21.779383 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:21.779356 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:21.779911 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:21.779460 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6rt6" podUID="a5428d1a-8c8b-41ce-ad97-a729ec7a2e43" Oct 30 12:51:21.779911 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:21.779503 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:21.779911 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:21.779560 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lf7b2" podUID="75bd9ac5-49ff-4c58-85c7-79b181150555" Oct 30 12:51:22.062681 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:22.062658 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" event={"ID":"23d95081-1188-4002-997e-65ca57111fab","Type":"ContainerStarted","Data":"d5ccfc5227a389540d51b7941c33f1a0e8507b5c6a09f556cd82dc4ccde8d018"} Oct 30 12:51:22.062951 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:22.062935 2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:22.063036 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:22.063002 2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:22.078341 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:22.078189 2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:22.163890 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:22.163846 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" podStartSLOduration=7.530000891 podStartE2EDuration="22.16383306s" totalImagesPullingTime="14.633832169s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 12:51:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:22.163803403 +0000 UTC m=+22.886606370" watchObservedRunningTime="2026-10-30 12:51:22.16383306 +0000 UTC m=+22.886636018" Oct 30 12:51:22.779201 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:22.779167 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:22.779357 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:22.779313 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24p2b" podUID="16a02457-151b-4e96-8118-e7695190b9be" Oct 30 12:51:23.065244 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:23.065219 2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:23.082246 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:23.082224 2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:23.198977 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:23.198943 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lf7b2"] Oct 30 12:51:23.199129 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:23.199081 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:23.199226 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:23.199203 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lf7b2" podUID="75bd9ac5-49ff-4c58-85c7-79b181150555" Oct 30 12:51:23.202110 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:23.202088 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-24p2b"] Oct 30 12:51:23.202216 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:23.202188 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:23.202323 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:23.202304 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24p2b" podUID="16a02457-151b-4e96-8118-e7695190b9be" Oct 30 12:51:23.202682 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:23.202665 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-d6rt6"] Oct 30 12:51:23.202747 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:23.202737 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:23.202808 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:23.202795 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6rt6" podUID="a5428d1a-8c8b-41ce-ad97-a729ec7a2e43" Oct 30 12:51:24.778699 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:24.778657 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:24.779178 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:24.778657 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:24.779178 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:24.778760 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6rt6" podUID="a5428d1a-8c8b-41ce-ad97-a729ec7a2e43" Oct 30 12:51:24.779178 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:24.778657 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:24.779178 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:24.778835 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24p2b" podUID="16a02457-151b-4e96-8118-e7695190b9be" Oct 30 12:51:24.779178 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:24.778926 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lf7b2" podUID="75bd9ac5-49ff-4c58-85c7-79b181150555" Oct 30 12:51:26.779025 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:26.778995 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:26.779559 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:26.779113 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lf7b2" podUID="75bd9ac5-49ff-4c58-85c7-79b181150555" Oct 30 12:51:26.779559 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:26.778997 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:26.779559 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:26.779285 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6rt6" podUID="a5428d1a-8c8b-41ce-ad97-a729ec7a2e43" Oct 30 12:51:26.779559 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:26.779318 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:26.779559 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:26.779393 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24p2b" podUID="16a02457-151b-4e96-8118-e7695190b9be" Oct 30 12:51:28.778513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:28.778482 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:28.778513 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:28.778508 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:28.778946 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:28.778482 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:28.778946 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:28.778588 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-24p2b" podUID="16a02457-151b-4e96-8118-e7695190b9be" Oct 30 12:51:28.778946 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:28.778659 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-lf7b2" podUID="75bd9ac5-49ff-4c58-85c7-79b181150555" Oct 30 12:51:28.778946 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:28.778733 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d6rt6" podUID="a5428d1a-8c8b-41ce-ad97-a729ec7a2e43" Oct 30 12:51:29.519429 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.519403 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeReady" Oct 30 12:51:29.519562 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.519526 2583 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Oct 30 12:51:29.577487 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.577456 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-nprbr"] Oct 30 12:51:29.596187 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.596163 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-nzp7p"] Oct 30 12:51:29.596319 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.596309 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-nprbr" Oct 30 12:51:29.606722 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.606696 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-wsfvm"] Oct 30 12:51:29.606844 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.606788 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-nzp7p" Oct 30 12:51:29.615546 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.615021 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp"] Oct 30 12:51:29.615546 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.615480 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-wsfvm" Oct 30 12:51:29.621060 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.621035 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\"" Oct 30 12:51:29.621326 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.621305 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\"" Oct 30 12:51:29.628624 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.628605 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-nbwj8"] Oct 30 12:51:29.628932 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.628803 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:29.638385 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.638364 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-bbb6d"] Oct 30 12:51:29.638506 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.638488 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:29.648731 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.648706 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn"] Oct 30 12:51:29.649263 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.649241 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:29.650475 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.650314 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Oct 30 12:51:29.659444 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.659422 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-bzh2m"] Oct 30 12:51:29.660363 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.660342 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:29.668826 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.668810 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5cfd489894-v5wlw"] Oct 30 12:51:29.668945 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.668929 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-bzh2m" Oct 30 12:51:29.677922 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.677906 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp"] Oct 30 12:51:29.678035 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.678021 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.684966 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.684951 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr"] Oct 30 12:51:29.685113 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.685099 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp" Oct 30 12:51:29.694816 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.694799 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-xsw6d"] Oct 30 12:51:29.694906 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.694890 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr" Oct 30 12:51:29.703748 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.703729 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-767cffcd47-qx67t"] Oct 30 12:51:29.703857 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.703847 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xsw6d" Oct 30 12:51:29.712671 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.712656 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5cd77977d7-wrk42"] Oct 30 12:51:29.712783 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.712770 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:29.720978 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.720962 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-49h5k"] Oct 30 12:51:29.721125 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.721108 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:29.721550 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.721530 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Oct 30 12:51:29.722030 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.722010 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Oct 30 12:51:29.722030 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.722027 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 12:51:29.722187 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.722056 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Oct 30 12:51:29.722187 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.722137 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Oct 30 12:51:29.722187 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.722015 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Oct 30 12:51:29.722187 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.722107 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Oct 30 12:51:29.722187 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.722114 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 12:51:29.722187 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.722102 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 12:51:29.722474 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.722192 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Oct 30 12:51:29.722474 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.722216 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Oct 30 12:51:29.722474 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.722228 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\"" Oct 30 12:51:29.722474 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.722236 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Oct 30 12:51:29.722474 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.722230 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Oct 30 12:51:29.722474 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.722260 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Oct 30 12:51:29.722474 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.722265 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Oct 30 12:51:29.722474 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.722298 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Oct 30 12:51:29.722474 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.722303 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Oct 30 12:51:29.729781 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.729766 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cd6w9"] Oct 30 12:51:29.729881 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.729866 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-49h5k" Oct 30 12:51:29.736701 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.736683 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 12:51:29.736887 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.736874 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Oct 30 12:51:29.737263 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.737249 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 12:51:29.737323 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.737234 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 12:51:29.738442 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.738425 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fnmsk"] Oct 30 12:51:29.738507 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.738491 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:29.747875 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.747857 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-nprbr"] Oct 30 12:51:29.747957 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.747895 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-wsfvm"] Oct 30 12:51:29.747957 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.747922 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-nzp7p"] Oct 30 12:51:29.747957 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.747946 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp"] Oct 30 12:51:29.748061 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.747969 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-nbwj8"] Oct 30 12:51:29.748061 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.747991 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-bbb6d"] Oct 30 12:51:29.748061 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.747947 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fnmsk" Oct 30 12:51:29.748061 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.748017 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cfd489894-v5wlw"] Oct 30 12:51:29.748061 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.748041 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-xsw6d"] Oct 30 12:51:29.748246 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.748077 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp"] Oct 30 12:51:29.748246 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.748101 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-767cffcd47-qx67t"] Oct 30 12:51:29.748246 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.748123 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fnmsk"] Oct 30 12:51:29.748246 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.748147 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cd6w9"] Oct 30 12:51:29.748246 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.748177 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cd77977d7-wrk42"] Oct 30 12:51:29.748246 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.748207 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn"] Oct 30 12:51:29.748246 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.748235 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-bzh2m"] Oct 30 12:51:29.748493 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.748256 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-49h5k"] Oct 30 12:51:29.748493 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.748286 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr"] Oct 30 12:51:29.751592 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.751575 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Oct 30 12:51:29.751687 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.751646 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Oct 30 12:51:29.751687 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.751679 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Oct 30 12:51:29.752810 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.752788 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Oct 30 12:51:29.753130 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.753111 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Oct 30 12:51:29.754177 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.754156 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Oct 30 12:51:29.754321 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.754300 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Oct 30 12:51:29.755809 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.755787 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-v2wwb\"" Oct 30 12:51:29.755901 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.755881 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-2x7bq\"" Oct 30 12:51:29.756189 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.756163 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Oct 30 12:51:29.756288 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.756272 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 12:51:29.756354 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.756344 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Oct 30 12:51:29.756404 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.756371 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-kzgpw\"" Oct 30 12:51:29.756509 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.756493 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Oct 30 12:51:29.756614 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.756599 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Oct 30 12:51:29.756667 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.756644 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-p6djf\"" Oct 30 12:51:29.756721 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.756687 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Oct 30 12:51:29.756894 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.756880 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-k7gdg\"" Oct 30 12:51:29.757044 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.757024 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-sgjxx\"" Oct 30 12:51:29.757147 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.757040 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Oct 30 12:51:29.757214 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.757146 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Oct 30 12:51:29.757361 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.757343 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-5t8jx\"" Oct 30 12:51:29.757429 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.757416 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cglsl\"" Oct 30 12:51:29.757545 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.757529 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Oct 30 12:51:29.757644 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.757585 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-xd4gf\"" Oct 30 12:51:29.757644 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.757610 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Oct 30 12:51:29.758007 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.757989 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Oct 30 12:51:29.758112 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.757999 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6jkd8\"" Oct 30 12:51:29.758185 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.758152 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 12:51:29.758601 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.758579 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-9lc5z\"" Oct 30 12:51:29.758992 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.758974 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ktwvb\"" Oct 30 12:51:29.759602 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.759581 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Oct 30 12:51:29.759905 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.759875 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Oct 30 12:51:29.764691 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.764672 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wtgzn\" (UID: \"66992144-51dd-4c1d-95c8-63862bca97cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:29.764780 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.764700 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c59v8\" (UniqueName: \"kubernetes.io/projected/fdddab43-acaf-4153-bb52-3e60044a35dc-kube-api-access-c59v8\") pod \"console-operator-7cd56b8dbc-nbwj8\" (UID: \"fdddab43-acaf-4153-bb52-3e60044a35dc\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:29.764780 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.764718 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fb047779-ad10-4f7e-a45a-4a154ab60ad8-trusted-ca\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.764780 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.764736 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scm4l\" (UniqueName: \"kubernetes.io/projected/f0d51a79-823b-4059-94c3-0535fc882a44-kube-api-access-scm4l\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:29.764922 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.764812 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75kpm\" (UniqueName: \"kubernetes.io/projected/2f5f56fc-d2d4-4cef-86c0-0ac4b7a51e41-kube-api-access-75kpm\") pod \"kas-connection-checker-685895c7c9-nprbr\" (UID: \"2f5f56fc-d2d4-4cef-86c0-0ac4b7a51e41\") " pod="kube-system/kas-connection-checker-685895c7c9-nprbr" Oct 30 12:51:29.764922 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.764869 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4v4t\" (UniqueName: \"kubernetes.io/projected/1734d386-0cb1-4002-a954-b76b3cf62eaf-kube-api-access-j4v4t\") pod \"kas-connection-checker-685895c7c9-wsfvm\" (UID: \"1734d386-0cb1-4002-a954-b76b3cf62eaf\") " pod="kube-system/kas-connection-checker-685895c7c9-wsfvm" Oct 30 12:51:29.764922 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.764899 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-tmp\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:29.764922 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.764914 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-snapshots\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:29.765136 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.764950 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.765136 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.765042 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fb047779-ad10-4f7e-a45a-4a154ab60ad8-ca-trust-extracted\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.765136 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.765088 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/52ca5902-69f0-462f-ae35-cbbc7f378345-config\") pod \"service-ca-operator-544d749c8c-fk7rp\" (UID: \"52ca5902-69f0-462f-ae35-cbbc7f378345\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp" Oct 30 12:51:29.765136 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.765118 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srvwc\" (UniqueName: \"kubernetes.io/projected/1a8f7b67-6d17-4f5e-a79b-52481ea0fd48-kube-api-access-srvwc\") pod \"volume-data-source-validator-58c9f5f46d-bzh2m\" (UID: \"1a8f7b67-6d17-4f5e-a79b-52481ea0fd48\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-bzh2m" Oct 30 12:51:29.765279 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.765144 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-wtgzn\" (UID: \"66992144-51dd-4c1d-95c8-63862bca97cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:29.765279 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.765200 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fdddab43-acaf-4153-bb52-3e60044a35dc-trusted-ca\") pod \"console-operator-7cd56b8dbc-nbwj8\" (UID: \"fdddab43-acaf-4153-bb52-3e60044a35dc\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:29.765723 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.765710 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f0d51a79-823b-4059-94c3-0535fc882a44-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:29.765761 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.765729 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg5w5\" (UniqueName: \"kubernetes.io/projected/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-kube-api-access-xg5w5\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:29.765761 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.765748 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7r5b\" (UniqueName: \"kubernetes.io/projected/52ca5902-69f0-462f-ae35-cbbc7f378345-kube-api-access-n7r5b\") pod \"service-ca-operator-544d749c8c-fk7rp\" (UID: \"52ca5902-69f0-462f-ae35-cbbc7f378345\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp" Oct 30 12:51:29.765842 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.765765 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-certificates\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.765842 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.765793 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk88t\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-kube-api-access-rk88t\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.765842 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.765833 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-service-ca-bundle\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:29.765961 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.765858 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:29.765961 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.765885 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/52ca5902-69f0-462f-ae35-cbbc7f378345-serving-cert\") pod \"service-ca-operator-544d749c8c-fk7rp\" (UID: \"52ca5902-69f0-462f-ae35-cbbc7f378345\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp" Oct 30 12:51:29.765961 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.765924 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fdddab43-acaf-4153-bb52-3e60044a35dc-config\") pod \"console-operator-7cd56b8dbc-nbwj8\" (UID: \"fdddab43-acaf-4153-bb52-3e60044a35dc\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:29.766058 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.765966 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fb047779-ad10-4f7e-a45a-4a154ab60ad8-installation-pull-secrets\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.766058 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.765985 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fdddab43-acaf-4153-bb52-3e60044a35dc-serving-cert\") pod \"console-operator-7cd56b8dbc-nbwj8\" (UID: \"fdddab43-acaf-4153-bb52-3e60044a35dc\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:29.766058 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.766022 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-serving-cert\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:29.766058 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.766045 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-bound-sa-token\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.766218 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.766126 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fb047779-ad10-4f7e-a45a-4a154ab60ad8-image-registry-private-configuration\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.766218 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.766144 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdmq4\" (UniqueName: \"kubernetes.io/projected/66992144-51dd-4c1d-95c8-63862bca97cc-kube-api-access-mdmq4\") pod \"cluster-samples-operator-6ddb6c8b94-wtgzn\" (UID: \"66992144-51dd-4c1d-95c8-63862bca97cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:29.766218 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.766171 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64858\" (UniqueName: \"kubernetes.io/projected/7d93802e-f47d-4319-b308-04fe84901551-kube-api-access-64858\") pod \"kas-connection-checker-685895c7c9-nzp7p\" (UID: \"7d93802e-f47d-4319-b308-04fe84901551\") " pod="kube-system/kas-connection-checker-685895c7c9-nzp7p" Oct 30 12:51:29.766218 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.766187 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:29.821873 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.821844 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 12:51:29.821873 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.821860 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 12:51:29.849035 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.849016 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-bhhxq\"" Oct 30 12:51:29.849035 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.849016 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 12:51:29.866443 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.866421 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-certificates\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:29.866539 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.866453 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fb047779-ad10-4f7e-a45a-4a154ab60ad8-trusted-ca\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.866539 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.866470 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scm4l\" (UniqueName: \"kubernetes.io/projected/f0d51a79-823b-4059-94c3-0535fc882a44-kube-api-access-scm4l\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:29.866539 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.866489 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75kpm\" (UniqueName: \"kubernetes.io/projected/2f5f56fc-d2d4-4cef-86c0-0ac4b7a51e41-kube-api-access-75kpm\") pod \"kas-connection-checker-685895c7c9-nprbr\" (UID: \"2f5f56fc-d2d4-4cef-86c0-0ac4b7a51e41\") " pod="kube-system/kas-connection-checker-685895c7c9-nprbr" Oct 30 12:51:29.866635 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.866595 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4v4t\" (UniqueName: \"kubernetes.io/projected/1734d386-0cb1-4002-a954-b76b3cf62eaf-kube-api-access-j4v4t\") pod \"kas-connection-checker-685895c7c9-wsfvm\" (UID: \"1734d386-0cb1-4002-a954-b76b3cf62eaf\") " pod="kube-system/kas-connection-checker-685895c7c9-wsfvm" Oct 30 12:51:29.866753 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.866734 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-tmp\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:29.866784 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.866768 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-snapshots\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:29.866815 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.866795 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-ca-trust-extracted\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:29.866923 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.866907 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.866994 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.866938 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-bound-sa-token\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:29.866994 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.866965 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-config-volume\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:29.867093 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867013 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8ksd\" (UniqueName: \"kubernetes.io/projected/0e13113b-b4d2-42d7-9e1c-642a790bcd55-kube-api-access-j8ksd\") pod \"network-check-source-6f78d59f99-xsw6d\" (UID: \"0e13113b-b4d2-42d7-9e1c-642a790bcd55\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xsw6d" Oct 30 12:51:29.867093 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:29.867048 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 12:51:29.867093 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:29.867088 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cfd489894-v5wlw: secret "image-registry-tls" not found Oct 30 12:51:29.867093 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867054 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjl5n\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-kube-api-access-gjl5n\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:29.867269 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:29.867168 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls podName:fb047779-ad10-4f7e-a45a-4a154ab60ad8 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:30.367148511 +0000 UTC m=+31.089951471 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls") pod "image-registry-5cfd489894-v5wlw" (UID: "fb047779-ad10-4f7e-a45a-4a154ab60ad8") : secret "image-registry-tls" not found Oct 30 12:51:29.867269 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867190 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-tmp-dir\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:29.867269 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867223 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fb047779-ad10-4f7e-a45a-4a154ab60ad8-ca-trust-extracted\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.867269 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867190 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-tmp\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:29.867269 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867249 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/52ca5902-69f0-462f-ae35-cbbc7f378345-config\") pod \"service-ca-operator-544d749c8c-fk7rp\" (UID: \"52ca5902-69f0-462f-ae35-cbbc7f378345\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp" Oct 30 12:51:29.867482 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867272 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srvwc\" (UniqueName: \"kubernetes.io/projected/1a8f7b67-6d17-4f5e-a79b-52481ea0fd48-kube-api-access-srvwc\") pod \"volume-data-source-validator-58c9f5f46d-bzh2m\" (UID: \"1a8f7b67-6d17-4f5e-a79b-52481ea0fd48\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-bzh2m" Oct 30 12:51:29.867482 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867296 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-wtgzn\" (UID: \"66992144-51dd-4c1d-95c8-63862bca97cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:29.867482 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867321 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fdddab43-acaf-4153-bb52-3e60044a35dc-trusted-ca\") pod \"console-operator-7cd56b8dbc-nbwj8\" (UID: \"fdddab43-acaf-4153-bb52-3e60044a35dc\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:29.867482 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867351 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f0d51a79-823b-4059-94c3-0535fc882a44-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:29.867482 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867386 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg5w5\" (UniqueName: \"kubernetes.io/projected/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-kube-api-access-xg5w5\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:29.867482 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867416 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/109e0336-ecdb-4278-b5d3-72c7bdb80276-nginx-conf\") pod \"networking-console-plugin-86c478bb88-49h5k\" (UID: \"109e0336-ecdb-4278-b5d3-72c7bdb80276\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-49h5k" Oct 30 12:51:29.867482 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867452 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7r5b\" (UniqueName: \"kubernetes.io/projected/52ca5902-69f0-462f-ae35-cbbc7f378345-kube-api-access-n7r5b\") pod \"service-ca-operator-544d749c8c-fk7rp\" (UID: \"52ca5902-69f0-462f-ae35-cbbc7f378345\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp" Oct 30 12:51:29.867482 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867467 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fb047779-ad10-4f7e-a45a-4a154ab60ad8-ca-trust-extracted\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.867482 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867481 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-certificates\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.867894 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867503 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk88t\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-kube-api-access-rk88t\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.867894 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867527 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-service-ca-bundle\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:29.867894 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867539 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-snapshots\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:29.867894 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867558 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:29.867894 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867596 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqfvq\" (UniqueName: \"kubernetes.io/projected/3e77ccc1-14a3-4f85-a214-854bd392733a-kube-api-access-pqfvq\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:29.867894 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867634 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/52ca5902-69f0-462f-ae35-cbbc7f378345-serving-cert\") pod \"service-ca-operator-544d749c8c-fk7rp\" (UID: \"52ca5902-69f0-462f-ae35-cbbc7f378345\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp" Oct 30 12:51:29.867894 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:29.867646 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 12:51:29.867894 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867663 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4ca5142-5b8a-4406-9ed0-06f9edea89bc-cert\") pod \"ingress-canary-fnmsk\" (UID: \"d4ca5142-5b8a-4406-9ed0-06f9edea89bc\") " pod="openshift-ingress-canary/ingress-canary-fnmsk" Oct 30 12:51:29.867894 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867691 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbwv9\" (UniqueName: \"kubernetes.io/projected/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-kube-api-access-sbwv9\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:29.867894 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:29.867718 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls podName:f0d51a79-823b-4059-94c3-0535fc882a44 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:30.367702083 +0000 UTC m=+31.090505044 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6s9dp" (UID: "f0d51a79-823b-4059-94c3-0535fc882a44") : secret "cluster-monitoring-operator-tls" not found Oct 30 12:51:29.867894 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867885 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/52ca5902-69f0-462f-ae35-cbbc7f378345-config\") pod \"service-ca-operator-544d749c8c-fk7rp\" (UID: \"52ca5902-69f0-462f-ae35-cbbc7f378345\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp" Oct 30 12:51:29.868406 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.867527 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fb047779-ad10-4f7e-a45a-4a154ab60ad8-trusted-ca\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.868406 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868057 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-wtgzn\" (UID: \"66992144-51dd-4c1d-95c8-63862bca97cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:29.868406 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868175 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fdddab43-acaf-4153-bb52-3e60044a35dc-config\") pod \"console-operator-7cd56b8dbc-nbwj8\" (UID: \"fdddab43-acaf-4153-bb52-3e60044a35dc\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:29.868406 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868224 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:29.868406 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868254 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-trusted-ca\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:29.868406 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868269 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f0d51a79-823b-4059-94c3-0535fc882a44-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:29.868406 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868290 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9951ec02-ee4f-488c-8cab-3985569328f8-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr\" (UID: \"9951ec02-ee4f-488c-8cab-3985569328f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr" Oct 30 12:51:29.868406 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868331 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-service-ca-bundle\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:29.868406 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868349 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-stats-auth\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:29.868406 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868368 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fdddab43-acaf-4153-bb52-3e60044a35dc-trusted-ca\") pod \"console-operator-7cd56b8dbc-nbwj8\" (UID: \"fdddab43-acaf-4153-bb52-3e60044a35dc\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:29.868406 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868385 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-metrics-tls\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:29.868903 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868501 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fb047779-ad10-4f7e-a45a-4a154ab60ad8-installation-pull-secrets\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.868903 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868568 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-image-registry-private-configuration\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:29.868903 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868619 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fdddab43-acaf-4153-bb52-3e60044a35dc-serving-cert\") pod \"console-operator-7cd56b8dbc-nbwj8\" (UID: \"fdddab43-acaf-4153-bb52-3e60044a35dc\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:29.868903 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868651 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-default-certificate\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:29.868903 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868673 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-certificates\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.868903 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868680 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-serving-cert\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:29.868903 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868705 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htqvt\" (UniqueName: \"kubernetes.io/projected/9951ec02-ee4f-488c-8cab-3985569328f8-kube-api-access-htqvt\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr\" (UID: \"9951ec02-ee4f-488c-8cab-3985569328f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr" Oct 30 12:51:29.868903 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868730 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fdddab43-acaf-4153-bb52-3e60044a35dc-config\") pod \"console-operator-7cd56b8dbc-nbwj8\" (UID: \"fdddab43-acaf-4153-bb52-3e60044a35dc\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:29.868903 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868803 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-bound-sa-token\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.868903 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868822 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fb047779-ad10-4f7e-a45a-4a154ab60ad8-image-registry-private-configuration\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.868903 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868851 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdmq4\" (UniqueName: \"kubernetes.io/projected/66992144-51dd-4c1d-95c8-63862bca97cc-kube-api-access-mdmq4\") pod \"cluster-samples-operator-6ddb6c8b94-wtgzn\" (UID: \"66992144-51dd-4c1d-95c8-63862bca97cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:29.868903 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868870 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64858\" (UniqueName: \"kubernetes.io/projected/7d93802e-f47d-4319-b308-04fe84901551-kube-api-access-64858\") pod \"kas-connection-checker-685895c7c9-nzp7p\" (UID: \"7d93802e-f47d-4319-b308-04fe84901551\") " pod="kube-system/kas-connection-checker-685895c7c9-nzp7p" Oct 30 12:51:29.868903 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868889 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9951ec02-ee4f-488c-8cab-3985569328f8-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr\" (UID: \"9951ec02-ee4f-488c-8cab-3985569328f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr" Oct 30 12:51:29.869425 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868911 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-installation-pull-secrets\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:29.869425 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868936 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:29.869425 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868962 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:29.869425 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.868987 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/109e0336-ecdb-4278-b5d3-72c7bdb80276-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-49h5k\" (UID: \"109e0336-ecdb-4278-b5d3-72c7bdb80276\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-49h5k" Oct 30 12:51:29.869425 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.869027 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wtgzn\" (UID: \"66992144-51dd-4c1d-95c8-63862bca97cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:29.869425 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.869061 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e77ccc1-14a3-4f85-a214-854bd392733a-service-ca-bundle\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:29.869425 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:29.869117 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 12:51:29.869425 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.869128 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h7bv\" (UniqueName: \"kubernetes.io/projected/d4ca5142-5b8a-4406-9ed0-06f9edea89bc-kube-api-access-5h7bv\") pod \"ingress-canary-fnmsk\" (UID: \"d4ca5142-5b8a-4406-9ed0-06f9edea89bc\") " pod="openshift-ingress-canary/ingress-canary-fnmsk" Oct 30 12:51:29.869425 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:29.869172 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls podName:66992144-51dd-4c1d-95c8-63862bca97cc nodeName:}" failed. No retries permitted until 2026-10-30 12:51:30.369155098 +0000 UTC m=+31.091958059 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-wtgzn" (UID: "66992144-51dd-4c1d-95c8-63862bca97cc") : secret "samples-operator-tls" not found Oct 30 12:51:29.869425 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.869191 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c59v8\" (UniqueName: \"kubernetes.io/projected/fdddab43-acaf-4153-bb52-3e60044a35dc-kube-api-access-c59v8\") pod \"console-operator-7cd56b8dbc-nbwj8\" (UID: \"fdddab43-acaf-4153-bb52-3e60044a35dc\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:29.869937 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.869915 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:29.871955 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.871935 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/52ca5902-69f0-462f-ae35-cbbc7f378345-serving-cert\") pod \"service-ca-operator-544d749c8c-fk7rp\" (UID: \"52ca5902-69f0-462f-ae35-cbbc7f378345\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp" Oct 30 12:51:29.872097 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.872002 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fdddab43-acaf-4153-bb52-3e60044a35dc-serving-cert\") pod \"console-operator-7cd56b8dbc-nbwj8\" (UID: \"fdddab43-acaf-4153-bb52-3e60044a35dc\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:29.872097 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.872047 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fb047779-ad10-4f7e-a45a-4a154ab60ad8-installation-pull-secrets\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.872213 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.872126 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-serving-cert\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:29.872213 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.872168 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fb047779-ad10-4f7e-a45a-4a154ab60ad8-image-registry-private-configuration\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.897169 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.897148 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75kpm\" (UniqueName: \"kubernetes.io/projected/2f5f56fc-d2d4-4cef-86c0-0ac4b7a51e41-kube-api-access-75kpm\") pod \"kas-connection-checker-685895c7c9-nprbr\" (UID: \"2f5f56fc-d2d4-4cef-86c0-0ac4b7a51e41\") " pod="kube-system/kas-connection-checker-685895c7c9-nprbr" Oct 30 12:51:29.899767 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.899748 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg5w5\" (UniqueName: \"kubernetes.io/projected/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-kube-api-access-xg5w5\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:29.906754 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.906717 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64858\" (UniqueName: \"kubernetes.io/projected/7d93802e-f47d-4319-b308-04fe84901551-kube-api-access-64858\") pod \"kas-connection-checker-685895c7c9-nzp7p\" (UID: \"7d93802e-f47d-4319-b308-04fe84901551\") " pod="kube-system/kas-connection-checker-685895c7c9-nzp7p" Oct 30 12:51:29.906838 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.906810 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdmq4\" (UniqueName: \"kubernetes.io/projected/66992144-51dd-4c1d-95c8-63862bca97cc-kube-api-access-mdmq4\") pod \"cluster-samples-operator-6ddb6c8b94-wtgzn\" (UID: \"66992144-51dd-4c1d-95c8-63862bca97cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:29.907158 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.907132 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7r5b\" (UniqueName: \"kubernetes.io/projected/52ca5902-69f0-462f-ae35-cbbc7f378345-kube-api-access-n7r5b\") pod \"service-ca-operator-544d749c8c-fk7rp\" (UID: \"52ca5902-69f0-462f-ae35-cbbc7f378345\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp" Oct 30 12:51:29.907270 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.907252 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srvwc\" (UniqueName: \"kubernetes.io/projected/1a8f7b67-6d17-4f5e-a79b-52481ea0fd48-kube-api-access-srvwc\") pod \"volume-data-source-validator-58c9f5f46d-bzh2m\" (UID: \"1a8f7b67-6d17-4f5e-a79b-52481ea0fd48\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-bzh2m" Oct 30 12:51:29.907587 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.907570 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4v4t\" (UniqueName: \"kubernetes.io/projected/1734d386-0cb1-4002-a954-b76b3cf62eaf-kube-api-access-j4v4t\") pod \"kas-connection-checker-685895c7c9-wsfvm\" (UID: \"1734d386-0cb1-4002-a954-b76b3cf62eaf\") " pod="kube-system/kas-connection-checker-685895c7c9-wsfvm" Oct 30 12:51:29.907774 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.907757 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c59v8\" (UniqueName: \"kubernetes.io/projected/fdddab43-acaf-4153-bb52-3e60044a35dc-kube-api-access-c59v8\") pod \"console-operator-7cd56b8dbc-nbwj8\" (UID: \"fdddab43-acaf-4153-bb52-3e60044a35dc\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:29.908579 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.908556 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-bound-sa-token\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.908991 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.908973 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk88t\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-kube-api-access-rk88t\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:29.911343 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.911323 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scm4l\" (UniqueName: \"kubernetes.io/projected/f0d51a79-823b-4059-94c3-0535fc882a44-kube-api-access-scm4l\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:29.919758 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.919741 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-nprbr" Oct 30 12:51:29.932987 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.932970 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-nzp7p" Oct 30 12:51:29.943898 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.943854 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-wsfvm" Oct 30 12:51:29.963775 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.963753 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:29.970498 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.970479 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-certificates\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:29.970591 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.970529 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-ca-trust-extracted\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:29.970591 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.970569 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-bound-sa-token\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:29.970717 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.970606 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-config-volume\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:29.970717 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.970660 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8ksd\" (UniqueName: \"kubernetes.io/projected/0e13113b-b4d2-42d7-9e1c-642a790bcd55-kube-api-access-j8ksd\") pod \"network-check-source-6f78d59f99-xsw6d\" (UID: \"0e13113b-b4d2-42d7-9e1c-642a790bcd55\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xsw6d" Oct 30 12:51:29.970717 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.970689 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjl5n\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-kube-api-access-gjl5n\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:29.970864 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.970716 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-tmp-dir\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:29.970864 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.970758 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/109e0336-ecdb-4278-b5d3-72c7bdb80276-nginx-conf\") pod \"networking-console-plugin-86c478bb88-49h5k\" (UID: \"109e0336-ecdb-4278-b5d3-72c7bdb80276\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-49h5k" Oct 30 12:51:29.970864 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.970809 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqfvq\" (UniqueName: \"kubernetes.io/projected/3e77ccc1-14a3-4f85-a214-854bd392733a-kube-api-access-pqfvq\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:29.970864 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.970836 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4ca5142-5b8a-4406-9ed0-06f9edea89bc-cert\") pod \"ingress-canary-fnmsk\" (UID: \"d4ca5142-5b8a-4406-9ed0-06f9edea89bc\") " pod="openshift-ingress-canary/ingress-canary-fnmsk" Oct 30 12:51:29.970864 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.970861 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbwv9\" (UniqueName: \"kubernetes.io/projected/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-kube-api-access-sbwv9\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:29.971138 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.970889 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:29.971138 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.970896 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-ca-trust-extracted\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:29.971138 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.970913 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-trusted-ca\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:29.971138 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.970945 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9951ec02-ee4f-488c-8cab-3985569328f8-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr\" (UID: \"9951ec02-ee4f-488c-8cab-3985569328f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr" Oct 30 12:51:29.971138 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.970970 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-stats-auth\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:29.971138 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.970992 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-metrics-tls\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:29.971138 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:29.970999 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 12:51:29.971138 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:29.971017 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cd77977d7-wrk42: secret "image-registry-tls" not found Oct 30 12:51:29.971138 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.971043 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-image-registry-private-configuration\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:29.971138 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:29.971092 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls podName:5133b471-c5f8-4b35-b06c-1d8f429c3ba2 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:30.471049483 +0000 UTC m=+31.193852432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls") pod "image-registry-5cd77977d7-wrk42" (UID: "5133b471-c5f8-4b35-b06c-1d8f429c3ba2") : secret "image-registry-tls" not found Oct 30 12:51:29.971138 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.971134 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-default-certificate\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:29.971690 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.971170 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htqvt\" (UniqueName: \"kubernetes.io/projected/9951ec02-ee4f-488c-8cab-3985569328f8-kube-api-access-htqvt\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr\" (UID: \"9951ec02-ee4f-488c-8cab-3985569328f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr" Oct 30 12:51:29.971690 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.971204 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-tmp-dir\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:29.971690 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.971216 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9951ec02-ee4f-488c-8cab-3985569328f8-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr\" (UID: \"9951ec02-ee4f-488c-8cab-3985569328f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr" Oct 30 12:51:29.971690 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.971246 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-installation-pull-secrets\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:29.971690 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.971088 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-certificates\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:29.971930 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.971834 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/109e0336-ecdb-4278-b5d3-72c7bdb80276-nginx-conf\") pod \"networking-console-plugin-86c478bb88-49h5k\" (UID: \"109e0336-ecdb-4278-b5d3-72c7bdb80276\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-49h5k" Oct 30 12:51:29.975105 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.972324 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-trusted-ca\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:29.975105 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.972362 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9951ec02-ee4f-488c-8cab-3985569328f8-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr\" (UID: \"9951ec02-ee4f-488c-8cab-3985569328f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr" Oct 30 12:51:29.975105 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:29.972556 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 12:51:29.975105 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.972591 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:29.975105 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:29.972609 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-metrics-tls podName:38b6bfaf-2fb2-455b-9e7d-32586b3f216f nodeName:}" failed. No retries permitted until 2026-10-30 12:51:30.47259644 +0000 UTC m=+31.195399386 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-metrics-tls") pod "dns-default-cd6w9" (UID: "38b6bfaf-2fb2-455b-9e7d-32586b3f216f") : secret "dns-default-metrics-tls" not found Oct 30 12:51:29.975105 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.972639 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/109e0336-ecdb-4278-b5d3-72c7bdb80276-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-49h5k\" (UID: \"109e0336-ecdb-4278-b5d3-72c7bdb80276\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-49h5k" Oct 30 12:51:29.975105 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:29.972660 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 12:51:29.975105 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.972683 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e77ccc1-14a3-4f85-a214-854bd392733a-service-ca-bundle\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:29.975105 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:29.972712 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs podName:3e77ccc1-14a3-4f85-a214-854bd392733a nodeName:}" failed. No retries permitted until 2026-10-30 12:51:30.472697924 +0000 UTC m=+31.195500886 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs") pod "router-default-767cffcd47-qx67t" (UID: "3e77ccc1-14a3-4f85-a214-854bd392733a") : secret "router-metrics-certs-default" not found Oct 30 12:51:29.975105 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.972731 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h7bv\" (UniqueName: \"kubernetes.io/projected/d4ca5142-5b8a-4406-9ed0-06f9edea89bc-kube-api-access-5h7bv\") pod \"ingress-canary-fnmsk\" (UID: \"d4ca5142-5b8a-4406-9ed0-06f9edea89bc\") " pod="openshift-ingress-canary/ingress-canary-fnmsk" Oct 30 12:51:29.975105 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:29.972763 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e77ccc1-14a3-4f85-a214-854bd392733a-service-ca-bundle podName:3e77ccc1-14a3-4f85-a214-854bd392733a nodeName:}" failed. No retries permitted until 2026-10-30 12:51:30.47275207 +0000 UTC m=+31.195555039 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3e77ccc1-14a3-4f85-a214-854bd392733a-service-ca-bundle") pod "router-default-767cffcd47-qx67t" (UID: "3e77ccc1-14a3-4f85-a214-854bd392733a") : configmap references non-existent config key: service-ca.crt Oct 30 12:51:29.975105 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:29.972825 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 12:51:29.975105 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:29.972858 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/109e0336-ecdb-4278-b5d3-72c7bdb80276-networking-console-plugin-cert podName:109e0336-ecdb-4278-b5d3-72c7bdb80276 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:30.472847953 +0000 UTC m=+31.195650899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/109e0336-ecdb-4278-b5d3-72c7bdb80276-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-49h5k" (UID: "109e0336-ecdb-4278-b5d3-72c7bdb80276") : secret "networking-console-plugin-cert" not found Oct 30 12:51:29.975105 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.973311 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:29.975105 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.973565 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-config-volume\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:29.975865 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.974253 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-image-registry-private-configuration\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:29.975865 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.974398 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4ca5142-5b8a-4406-9ed0-06f9edea89bc-cert\") pod \"ingress-canary-fnmsk\" (UID: \"d4ca5142-5b8a-4406-9ed0-06f9edea89bc\") " pod="openshift-ingress-canary/ingress-canary-fnmsk" Oct 30 12:51:29.975865 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.974828 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-stats-auth\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:29.975865 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.974939 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-default-certificate\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:29.975865 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.975194 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9951ec02-ee4f-488c-8cab-3985569328f8-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr\" (UID: \"9951ec02-ee4f-488c-8cab-3985569328f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr" Oct 30 12:51:29.975865 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.975593 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-installation-pull-secrets\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:29.999766 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:29.999703 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-bzh2m" Oct 30 12:51:30.004545 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.004511 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-bound-sa-token\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:30.010643 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.010511 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp" Oct 30 12:51:30.012347 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.012323 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqfvq\" (UniqueName: \"kubernetes.io/projected/3e77ccc1-14a3-4f85-a214-854bd392733a-kube-api-access-pqfvq\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:30.013522 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.012808 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htqvt\" (UniqueName: \"kubernetes.io/projected/9951ec02-ee4f-488c-8cab-3985569328f8-kube-api-access-htqvt\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr\" (UID: \"9951ec02-ee4f-488c-8cab-3985569328f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr" Oct 30 12:51:30.014683 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.014189 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h7bv\" (UniqueName: \"kubernetes.io/projected/d4ca5142-5b8a-4406-9ed0-06f9edea89bc-kube-api-access-5h7bv\") pod \"ingress-canary-fnmsk\" (UID: \"d4ca5142-5b8a-4406-9ed0-06f9edea89bc\") " pod="openshift-ingress-canary/ingress-canary-fnmsk" Oct 30 12:51:30.014683 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.014646 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjl5n\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-kube-api-access-gjl5n\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:30.015976 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.015914 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbwv9\" (UniqueName: \"kubernetes.io/projected/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-kube-api-access-sbwv9\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:30.020265 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.020241 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8ksd\" (UniqueName: \"kubernetes.io/projected/0e13113b-b4d2-42d7-9e1c-642a790bcd55-kube-api-access-j8ksd\") pod \"network-check-source-6f78d59f99-xsw6d\" (UID: \"0e13113b-b4d2-42d7-9e1c-642a790bcd55\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xsw6d" Oct 30 12:51:30.026240 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.026042 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xsw6d" Oct 30 12:51:30.035295 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.035277 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr" Oct 30 12:51:30.077795 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.077737 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fnmsk" Oct 30 12:51:30.325649 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.325611 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-nprbr"] Oct 30 12:51:30.329512 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:51:30.329488 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f5f56fc_d2d4_4cef_86c0_0ac4b7a51e41.slice/crio-ee279992e3ac50ef6da1a2f63883c9d08d538b348a1eb641a44e81c9aa3da5fc WatchSource:0}: Error finding container ee279992e3ac50ef6da1a2f63883c9d08d538b348a1eb641a44e81c9aa3da5fc: Status 404 returned error can't find the container with id ee279992e3ac50ef6da1a2f63883c9d08d538b348a1eb641a44e81c9aa3da5fc Oct 30 12:51:30.331281 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.331253 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-nzp7p"] Oct 30 12:51:30.333676 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:51:30.333605 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d93802e_f47d_4319_b308_04fe84901551.slice/crio-64bdaa665cabf5cc4b070b756ef76602bdfeadbc86d142af3996b25b23e652cd WatchSource:0}: Error finding container 64bdaa665cabf5cc4b070b756ef76602bdfeadbc86d142af3996b25b23e652cd: Status 404 returned error can't find the container with id 64bdaa665cabf5cc4b070b756ef76602bdfeadbc86d142af3996b25b23e652cd Oct 30 12:51:30.344772 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.344751 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-wsfvm"] Oct 30 12:51:30.349360 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.348191 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-bzh2m"] Oct 30 12:51:30.353466 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.353371 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-bbb6d"] Oct 30 12:51:30.355092 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.354640 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-nbwj8"] Oct 30 12:51:30.357156 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:51:30.357133 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9547ad8f_4eb0_48e9_a1ed_8a5efe0d5462.slice/crio-5d9c63e84cebf9646cbd5eebebf0de9fd16426bee0278ac370280e367efef2c6 WatchSource:0}: Error finding container 5d9c63e84cebf9646cbd5eebebf0de9fd16426bee0278ac370280e367efef2c6: Status 404 returned error can't find the container with id 5d9c63e84cebf9646cbd5eebebf0de9fd16426bee0278ac370280e367efef2c6 Oct 30 12:51:30.366133 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:51:30.366104 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdddab43_acaf_4153_bb52_3e60044a35dc.slice/crio-f7bd715f51c9289bcb3cd2d9c8f31ed55e2817493c4c2ad5e461b98dd43336fe WatchSource:0}: Error finding container f7bd715f51c9289bcb3cd2d9c8f31ed55e2817493c4c2ad5e461b98dd43336fe: Status 404 returned error can't find the container with id f7bd715f51c9289bcb3cd2d9c8f31ed55e2817493c4c2ad5e461b98dd43336fe Oct 30 12:51:30.367207 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:51:30.366610 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a8f7b67_6d17_4f5e_a79b_52481ea0fd48.slice/crio-c305a6af368aa3db922feb23caa6802fb98ab51beca667b418ee32ea7d4c79dd WatchSource:0}: Error finding container c305a6af368aa3db922feb23caa6802fb98ab51beca667b418ee32ea7d4c79dd: Status 404 returned error can't find the container with id c305a6af368aa3db922feb23caa6802fb98ab51beca667b418ee32ea7d4c79dd Oct 30 12:51:30.378815 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.378679 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:30.379088 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.379025 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wtgzn\" (UID: \"66992144-51dd-4c1d-95c8-63862bca97cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:30.379326 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.379276 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:30.379842 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:30.379561 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 12:51:30.379842 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:30.379580 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cfd489894-v5wlw: secret "image-registry-tls" not found Oct 30 12:51:30.379842 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:30.379628 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls podName:fb047779-ad10-4f7e-a45a-4a154ab60ad8 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:31.379610396 +0000 UTC m=+32.102413362 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls") pod "image-registry-5cfd489894-v5wlw" (UID: "fb047779-ad10-4f7e-a45a-4a154ab60ad8") : secret "image-registry-tls" not found Oct 30 12:51:30.379842 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:30.379691 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 12:51:30.379842 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:30.379727 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls podName:f0d51a79-823b-4059-94c3-0535fc882a44 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:31.379715 +0000 UTC m=+32.102517966 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6s9dp" (UID: "f0d51a79-823b-4059-94c3-0535fc882a44") : secret "cluster-monitoring-operator-tls" not found Oct 30 12:51:30.379842 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:30.379793 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 12:51:30.379842 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:30.379826 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls podName:66992144-51dd-4c1d-95c8-63862bca97cc nodeName:}" failed. No retries permitted until 2026-10-30 12:51:31.379815806 +0000 UTC m=+32.102618769 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-wtgzn" (UID: "66992144-51dd-4c1d-95c8-63862bca97cc") : secret "samples-operator-tls" not found Oct 30 12:51:30.413590 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.412731 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp"] Oct 30 12:51:30.427354 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.427325 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-xsw6d"] Oct 30 12:51:30.431544 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.431475 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr"] Oct 30 12:51:30.432779 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:51:30.432748 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e13113b_b4d2_42d7_9e1c_642a790bcd55.slice/crio-9231d3aca1a5eada24e65c5d97f815cd61a12bf38d58e2285a9c0a06d074d3da WatchSource:0}: Error finding container 9231d3aca1a5eada24e65c5d97f815cd61a12bf38d58e2285a9c0a06d074d3da: Status 404 returned error can't find the container with id 9231d3aca1a5eada24e65c5d97f815cd61a12bf38d58e2285a9c0a06d074d3da Oct 30 12:51:30.436340 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:51:30.436313 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9951ec02_ee4f_488c_8cab_3985569328f8.slice/crio-20283987a635d95c8db1514b2f3ef49bf514699f23b3ad259d918ae38cde43f5 WatchSource:0}: Error finding container 20283987a635d95c8db1514b2f3ef49bf514699f23b3ad259d918ae38cde43f5: Status 404 returned error can't find the container with id 20283987a635d95c8db1514b2f3ef49bf514699f23b3ad259d918ae38cde43f5 Oct 30 12:51:30.467768 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.466446 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fnmsk"] Oct 30 12:51:30.477888 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:51:30.477861 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4ca5142_5b8a_4406_9ed0_06f9edea89bc.slice/crio-8a211458f90a22000e7c1f0683391f60671e19098dee129dbfbc3fee70f7a3b0 WatchSource:0}: Error finding container 8a211458f90a22000e7c1f0683391f60671e19098dee129dbfbc3fee70f7a3b0: Status 404 returned error can't find the container with id 8a211458f90a22000e7c1f0683391f60671e19098dee129dbfbc3fee70f7a3b0 Oct 30 12:51:30.480797 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.480773 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:30.480875 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.480806 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-metrics-tls\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:30.480875 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.480846 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:30.480937 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.480884 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/109e0336-ecdb-4278-b5d3-72c7bdb80276-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-49h5k\" (UID: \"109e0336-ecdb-4278-b5d3-72c7bdb80276\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-49h5k" Oct 30 12:51:30.480937 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.480926 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e77ccc1-14a3-4f85-a214-854bd392733a-service-ca-bundle\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:30.480995 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:30.480934 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 12:51:30.480995 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:30.480944 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 12:51:30.480995 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:30.480963 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cd77977d7-wrk42: secret "image-registry-tls" not found Oct 30 12:51:30.480995 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:30.480984 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-metrics-tls podName:38b6bfaf-2fb2-455b-9e7d-32586b3f216f nodeName:}" failed. No retries permitted until 2026-10-30 12:51:31.480966198 +0000 UTC m=+32.203769162 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-metrics-tls") pod "dns-default-cd6w9" (UID: "38b6bfaf-2fb2-455b-9e7d-32586b3f216f") : secret "dns-default-metrics-tls" not found Oct 30 12:51:30.481137 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:30.481015 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls podName:5133b471-c5f8-4b35-b06c-1d8f429c3ba2 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:31.480998447 +0000 UTC m=+32.203801414 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls") pod "image-registry-5cd77977d7-wrk42" (UID: "5133b471-c5f8-4b35-b06c-1d8f429c3ba2") : secret "image-registry-tls" not found Oct 30 12:51:30.481137 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:30.481023 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 12:51:30.481137 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:30.481043 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e77ccc1-14a3-4f85-a214-854bd392733a-service-ca-bundle podName:3e77ccc1-14a3-4f85-a214-854bd392733a nodeName:}" failed. No retries permitted until 2026-10-30 12:51:31.481030705 +0000 UTC m=+32.203833666 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3e77ccc1-14a3-4f85-a214-854bd392733a-service-ca-bundle") pod "router-default-767cffcd47-qx67t" (UID: "3e77ccc1-14a3-4f85-a214-854bd392733a") : configmap references non-existent config key: service-ca.crt Oct 30 12:51:30.481137 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:30.481057 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 12:51:30.481137 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:30.481079 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs podName:3e77ccc1-14a3-4f85-a214-854bd392733a nodeName:}" failed. No retries permitted until 2026-10-30 12:51:31.481054344 +0000 UTC m=+32.203857292 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs") pod "router-default-767cffcd47-qx67t" (UID: "3e77ccc1-14a3-4f85-a214-854bd392733a") : secret "router-metrics-certs-default" not found Oct 30 12:51:30.481137 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:30.481110 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/109e0336-ecdb-4278-b5d3-72c7bdb80276-networking-console-plugin-cert podName:109e0336-ecdb-4278-b5d3-72c7bdb80276 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:31.481098027 +0000 UTC m=+32.203900992 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/109e0336-ecdb-4278-b5d3-72c7bdb80276-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-49h5k" (UID: "109e0336-ecdb-4278-b5d3-72c7bdb80276") : secret "networking-console-plugin-cert" not found Oct 30 12:51:30.778901 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.778741 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:30.778901 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.778757 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:30.779124 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.778971 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:30.797413 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.797392 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 12:51:30.797830 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.797807 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 12:51:30.797920 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.797855 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-68fxr\"" Oct 30 12:51:30.798102 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.798084 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-wqgtx\"" Oct 30 12:51:30.798210 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:30.798161 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hszmd\"" Oct 30 12:51:31.090904 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.090862 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" event={"ID":"fdddab43-acaf-4153-bb52-3e60044a35dc","Type":"ContainerStarted","Data":"f7bd715f51c9289bcb3cd2d9c8f31ed55e2817493c4c2ad5e461b98dd43336fe"} Oct 30 12:51:31.094954 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.094925 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fnmsk" event={"ID":"d4ca5142-5b8a-4406-9ed0-06f9edea89bc","Type":"ContainerStarted","Data":"8a211458f90a22000e7c1f0683391f60671e19098dee129dbfbc3fee70f7a3b0"} Oct 30 12:51:31.096227 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.096199 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp" event={"ID":"52ca5902-69f0-462f-ae35-cbbc7f378345","Type":"ContainerStarted","Data":"0aad8a9e73dea6fa13518239e610366e206fdf55a94179fa84e3f08d34e96599"} Oct 30 12:51:31.103056 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.103028 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-bzh2m" event={"ID":"1a8f7b67-6d17-4f5e-a79b-52481ea0fd48","Type":"ContainerStarted","Data":"c305a6af368aa3db922feb23caa6802fb98ab51beca667b418ee32ea7d4c79dd"} Oct 30 12:51:31.109089 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.108634 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-nprbr" event={"ID":"2f5f56fc-d2d4-4cef-86c0-0ac4b7a51e41","Type":"ContainerStarted","Data":"1181d3b055cfa79dc6f0f8cb32af03f8a18bcb2cf72e5c750a21e3a14c492698"} Oct 30 12:51:31.109089 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.108676 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-nprbr" event={"ID":"2f5f56fc-d2d4-4cef-86c0-0ac4b7a51e41","Type":"ContainerStarted","Data":"ee279992e3ac50ef6da1a2f63883c9d08d538b348a1eb641a44e81c9aa3da5fc"} Oct 30 12:51:31.110906 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.110829 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-bbb6d" event={"ID":"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462","Type":"ContainerStarted","Data":"5d9c63e84cebf9646cbd5eebebf0de9fd16426bee0278ac370280e367efef2c6"} Oct 30 12:51:31.112584 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.112560 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-nzp7p" event={"ID":"7d93802e-f47d-4319-b308-04fe84901551","Type":"ContainerStarted","Data":"430e51b1b5e7f6fda19696a30dd27ed422a43622f43f52e68f7b30ebc74f3f42"} Oct 30 12:51:31.112700 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.112603 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-nzp7p" event={"ID":"7d93802e-f47d-4319-b308-04fe84901551","Type":"ContainerStarted","Data":"64bdaa665cabf5cc4b070b756ef76602bdfeadbc86d142af3996b25b23e652cd"} Oct 30 12:51:31.115344 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.115276 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr" event={"ID":"9951ec02-ee4f-488c-8cab-3985569328f8","Type":"ContainerStarted","Data":"20283987a635d95c8db1514b2f3ef49bf514699f23b3ad259d918ae38cde43f5"} Oct 30 12:51:31.116651 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.116584 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xsw6d" event={"ID":"0e13113b-b4d2-42d7-9e1c-642a790bcd55","Type":"ContainerStarted","Data":"9231d3aca1a5eada24e65c5d97f815cd61a12bf38d58e2285a9c0a06d074d3da"} Oct 30 12:51:31.118953 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.118509 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-wsfvm" event={"ID":"1734d386-0cb1-4002-a954-b76b3cf62eaf","Type":"ContainerStarted","Data":"15e78c5cc314852462f49f4a9d559797743b62de354d754b5616919be5352fc7"} Oct 30 12:51:31.118953 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.118547 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-wsfvm" event={"ID":"1734d386-0cb1-4002-a954-b76b3cf62eaf","Type":"ContainerStarted","Data":"6cf91b452eede6076a34dcea2736ad13d357a8050ee4e321a6c9ae82f3890534"} Oct 30 12:51:31.155499 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.155192 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-nprbr" podStartSLOduration=375.155174077 podStartE2EDuration="6m15.155174077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:45:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:31.153435716 +0000 UTC m=+31.876238697" watchObservedRunningTime="2026-10-30 12:51:31.155174077 +0000 UTC m=+31.877977048" Oct 30 12:51:31.201329 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.200684 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-nzp7p" podStartSLOduration=375.200665418 podStartE2EDuration="6m15.200665418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:45:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:31.199377172 +0000 UTC m=+31.922180141" watchObservedRunningTime="2026-10-30 12:51:31.200665418 +0000 UTC m=+31.923468386" Oct 30 12:51:31.391276 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.391200 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wtgzn\" (UID: \"66992144-51dd-4c1d-95c8-63862bca97cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:31.391432 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.391292 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:31.391432 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:31.391338 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 12:51:31.391432 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.391353 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:31.391571 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:31.391430 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 12:51:31.391571 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:31.391445 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cfd489894-v5wlw: secret "image-registry-tls" not found Oct 30 12:51:31.391571 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:31.391489 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls podName:fb047779-ad10-4f7e-a45a-4a154ab60ad8 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:33.3914698 +0000 UTC m=+34.114272759 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls") pod "image-registry-5cfd489894-v5wlw" (UID: "fb047779-ad10-4f7e-a45a-4a154ab60ad8") : secret "image-registry-tls" not found Oct 30 12:51:31.391571 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:31.391506 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls podName:66992144-51dd-4c1d-95c8-63862bca97cc nodeName:}" failed. No retries permitted until 2026-10-30 12:51:33.391497471 +0000 UTC m=+34.114300422 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-wtgzn" (UID: "66992144-51dd-4c1d-95c8-63862bca97cc") : secret "samples-operator-tls" not found Oct 30 12:51:31.391571 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:31.391554 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 12:51:31.391815 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:31.391607 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls podName:f0d51a79-823b-4059-94c3-0535fc882a44 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:33.39159127 +0000 UTC m=+34.114394241 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6s9dp" (UID: "f0d51a79-823b-4059-94c3-0535fc882a44") : secret "cluster-monitoring-operator-tls" not found Oct 30 12:51:31.492339 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.492295 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:31.492339 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.492342 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-metrics-tls\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:31.492561 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.492400 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:31.492561 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.492428 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/109e0336-ecdb-4278-b5d3-72c7bdb80276-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-49h5k\" (UID: \"109e0336-ecdb-4278-b5d3-72c7bdb80276\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-49h5k" Oct 30 12:51:31.492561 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:31.492467 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e77ccc1-14a3-4f85-a214-854bd392733a-service-ca-bundle\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:31.493257 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:31.492698 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e77ccc1-14a3-4f85-a214-854bd392733a-service-ca-bundle podName:3e77ccc1-14a3-4f85-a214-854bd392733a nodeName:}" failed. No retries permitted until 2026-10-30 12:51:33.492678785 +0000 UTC m=+34.215481734 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3e77ccc1-14a3-4f85-a214-854bd392733a-service-ca-bundle") pod "router-default-767cffcd47-qx67t" (UID: "3e77ccc1-14a3-4f85-a214-854bd392733a") : configmap references non-existent config key: service-ca.crt Oct 30 12:51:31.493257 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:31.493084 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 12:51:31.493257 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:31.493100 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cd77977d7-wrk42: secret "image-registry-tls" not found Oct 30 12:51:31.493257 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:31.493147 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls podName:5133b471-c5f8-4b35-b06c-1d8f429c3ba2 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:33.493130597 +0000 UTC m=+34.215933544 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls") pod "image-registry-5cd77977d7-wrk42" (UID: "5133b471-c5f8-4b35-b06c-1d8f429c3ba2") : secret "image-registry-tls" not found Oct 30 12:51:31.493257 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:31.493172 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 12:51:31.493257 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:31.493207 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 12:51:31.493257 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:31.493212 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs podName:3e77ccc1-14a3-4f85-a214-854bd392733a nodeName:}" failed. No retries permitted until 2026-10-30 12:51:33.493200373 +0000 UTC m=+34.216003333 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs") pod "router-default-767cffcd47-qx67t" (UID: "3e77ccc1-14a3-4f85-a214-854bd392733a") : secret "router-metrics-certs-default" not found Oct 30 12:51:31.493257 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:31.493219 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 12:51:31.493257 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:31.493238 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-metrics-tls podName:38b6bfaf-2fb2-455b-9e7d-32586b3f216f nodeName:}" failed. No retries permitted until 2026-10-30 12:51:33.493227978 +0000 UTC m=+34.216030926 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-metrics-tls") pod "dns-default-cd6w9" (UID: "38b6bfaf-2fb2-455b-9e7d-32586b3f216f") : secret "dns-default-metrics-tls" not found Oct 30 12:51:31.493739 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:31.493288 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/109e0336-ecdb-4278-b5d3-72c7bdb80276-networking-console-plugin-cert podName:109e0336-ecdb-4278-b5d3-72c7bdb80276 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:33.493270568 +0000 UTC m=+34.216073531 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/109e0336-ecdb-4278-b5d3-72c7bdb80276-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-49h5k" (UID: "109e0336-ecdb-4278-b5d3-72c7bdb80276") : secret "networking-console-plugin-cert" not found Oct 30 12:51:33.131924 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:33.131147 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-bzh2m" event={"ID":"1a8f7b67-6d17-4f5e-a79b-52481ea0fd48","Type":"ContainerStarted","Data":"7c54db50bab0f1fcf29319b12cae75e5597ef4ecea46113fd3fe8a67d217799e"} Oct 30 12:51:33.177966 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:33.177893 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-wsfvm" podStartSLOduration=377.177872326 podStartE2EDuration="6m17.177872326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:45:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:31.249408696 +0000 UTC m=+31.972211665" watchObservedRunningTime="2026-10-30 12:51:33.177872326 +0000 UTC m=+33.900675298" Oct 30 12:51:33.179035 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:33.178992 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-bzh2m" podStartSLOduration=340.970549939 podStartE2EDuration="5m43.178982059s" totalImagesPullingTime="2.20843212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:45:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:33.176875338 +0000 UTC m=+33.899678309" watchObservedRunningTime="2026-10-30 12:51:33.178982059 +0000 UTC m=+33.901785028" Oct 30 12:51:33.416806 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:33.415811 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wtgzn\" (UID: \"66992144-51dd-4c1d-95c8-63862bca97cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:33.416806 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:33.415899 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:33.416806 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:33.415958 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:33.416806 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:33.416116 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 12:51:33.416806 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:33.416180 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls podName:f0d51a79-823b-4059-94c3-0535fc882a44 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:37.416161686 +0000 UTC m=+38.138964633 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6s9dp" (UID: "f0d51a79-823b-4059-94c3-0535fc882a44") : secret "cluster-monitoring-operator-tls" not found Oct 30 12:51:33.416806 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:33.416567 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 12:51:33.416806 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:33.416620 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls podName:66992144-51dd-4c1d-95c8-63862bca97cc nodeName:}" failed. No retries permitted until 2026-10-30 12:51:37.416605389 +0000 UTC m=+38.139408335 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-wtgzn" (UID: "66992144-51dd-4c1d-95c8-63862bca97cc") : secret "samples-operator-tls" not found Oct 30 12:51:33.416806 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:33.416684 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 12:51:33.416806 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:33.416697 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cfd489894-v5wlw: secret "image-registry-tls" not found Oct 30 12:51:33.416806 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:33.416728 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls podName:fb047779-ad10-4f7e-a45a-4a154ab60ad8 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:37.416717234 +0000 UTC m=+38.139520184 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls") pod "image-registry-5cfd489894-v5wlw" (UID: "fb047779-ad10-4f7e-a45a-4a154ab60ad8") : secret "image-registry-tls" not found Oct 30 12:51:33.518624 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:33.516905 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:33.518624 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:33.516949 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/109e0336-ecdb-4278-b5d3-72c7bdb80276-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-49h5k\" (UID: \"109e0336-ecdb-4278-b5d3-72c7bdb80276\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-49h5k" Oct 30 12:51:33.518624 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:33.516986 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e77ccc1-14a3-4f85-a214-854bd392733a-service-ca-bundle\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:33.518624 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:33.517013 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs\") pod \"network-metrics-daemon-24p2b\" (UID: \"16a02457-151b-4e96-8118-e7695190b9be\") " pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:33.518624 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:33.517088 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:33.518624 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:33.517121 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-metrics-tls\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:33.518624 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:33.517168 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:33.518624 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:33.517794 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e77ccc1-14a3-4f85-a214-854bd392733a-service-ca-bundle podName:3e77ccc1-14a3-4f85-a214-854bd392733a nodeName:}" failed. No retries permitted until 2026-10-30 12:51:37.517778426 +0000 UTC m=+38.240581372 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3e77ccc1-14a3-4f85-a214-854bd392733a-service-ca-bundle") pod "router-default-767cffcd47-qx67t" (UID: "3e77ccc1-14a3-4f85-a214-854bd392733a") : configmap references non-existent config key: service-ca.crt Oct 30 12:51:33.518624 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:33.517858 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 12:51:33.518624 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:33.517883 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs podName:3e77ccc1-14a3-4f85-a214-854bd392733a nodeName:}" failed. No retries permitted until 2026-10-30 12:51:37.517876061 +0000 UTC m=+38.240679007 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs") pod "router-default-767cffcd47-qx67t" (UID: "3e77ccc1-14a3-4f85-a214-854bd392733a") : secret "router-metrics-certs-default" not found Oct 30 12:51:33.518624 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:33.517920 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 12:51:33.518624 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:33.517960 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/109e0336-ecdb-4278-b5d3-72c7bdb80276-networking-console-plugin-cert podName:109e0336-ecdb-4278-b5d3-72c7bdb80276 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:37.517941374 +0000 UTC m=+38.240744324 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/109e0336-ecdb-4278-b5d3-72c7bdb80276-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-49h5k" (UID: "109e0336-ecdb-4278-b5d3-72c7bdb80276") : secret "networking-console-plugin-cert" not found Oct 30 12:51:33.518624 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:33.518351 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 12:51:33.518624 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:33.518367 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cd77977d7-wrk42: secret "image-registry-tls" not found Oct 30 12:51:33.518624 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:33.518412 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls podName:5133b471-c5f8-4b35-b06c-1d8f429c3ba2 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:37.518396821 +0000 UTC m=+38.241199769 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls") pod "image-registry-5cd77977d7-wrk42" (UID: "5133b471-c5f8-4b35-b06c-1d8f429c3ba2") : secret "image-registry-tls" not found Oct 30 12:51:33.518624 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:33.518478 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Oct 30 12:51:33.519509 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:33.518509 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs podName:16a02457-151b-4e96-8118-e7695190b9be nodeName:}" failed. No retries permitted until 2026-10-30 12:52:05.518499146 +0000 UTC m=+66.241302092 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs") pod "network-metrics-daemon-24p2b" (UID: "16a02457-151b-4e96-8118-e7695190b9be") : secret "metrics-daemon-secret" not found Oct 30 12:51:33.519509 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:33.518570 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 12:51:33.519509 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:33.518600 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-metrics-tls podName:38b6bfaf-2fb2-455b-9e7d-32586b3f216f nodeName:}" failed. No retries permitted until 2026-10-30 12:51:37.518590519 +0000 UTC m=+38.241393467 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-metrics-tls") pod "dns-default-cd6w9" (UID: "38b6bfaf-2fb2-455b-9e7d-32586b3f216f") : secret "dns-default-metrics-tls" not found Oct 30 12:51:33.527951 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:33.522591 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:33.527951 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:33.527644 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:33.618637 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:33.618159 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6zpc\" (UniqueName: \"kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc\") pod \"network-check-target-d6rt6\" (UID: \"a5428d1a-8c8b-41ce-ad97-a729ec7a2e43\") " pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:33.622739 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:33.622711 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6zpc\" (UniqueName: \"kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc\") pod \"network-check-target-d6rt6\" (UID: \"a5428d1a-8c8b-41ce-ad97-a729ec7a2e43\") " pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:33.808554 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:33.808221 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:33.904905 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:33.904871 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-lf7b2"] Oct 30 12:51:33.909114 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:51:33.908521 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75bd9ac5_49ff_4c58_85c7_79b181150555.slice/crio-68be897524ba08439268cdfb7db77cfeb418946d1e58c1c8faca16f7e39ac5bb WatchSource:0}: Error finding container 68be897524ba08439268cdfb7db77cfeb418946d1e58c1c8faca16f7e39ac5bb: Status 404 returned error can't find the container with id 68be897524ba08439268cdfb7db77cfeb418946d1e58c1c8faca16f7e39ac5bb Oct 30 12:51:34.136416 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:34.136373 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-lf7b2" event={"ID":"75bd9ac5-49ff-4c58-85c7-79b181150555","Type":"ContainerStarted","Data":"68be897524ba08439268cdfb7db77cfeb418946d1e58c1c8faca16f7e39ac5bb"} Oct 30 12:51:34.189097 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:34.189011 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-d6rt6"] Oct 30 12:51:34.369788 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:51:34.365303 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5428d1a_8c8b_41ce_ad97_a729ec7a2e43.slice/crio-0c0a7e5307867f709ccc4a87d9633f02f62862e18e356e3011abb8924c16d1db WatchSource:0}: Error finding container 0c0a7e5307867f709ccc4a87d9633f02f62862e18e356e3011abb8924c16d1db: Status 404 returned error can't find the container with id 0c0a7e5307867f709ccc4a87d9633f02f62862e18e356e3011abb8924c16d1db Oct 30 12:51:35.148646 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.148626 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/0.log" Oct 30 12:51:35.149031 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.148664 2583 generic.go:356] "Generic (PLEG): container finished" podID="fdddab43-acaf-4153-bb52-3e60044a35dc" containerID="ef437c5b6668718aaba07f77a5bb583d28dc3c5f1b9f2a41e66829415b554032" exitCode=255 Oct 30 12:51:35.149031 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.148767 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" event={"ID":"fdddab43-acaf-4153-bb52-3e60044a35dc","Type":"ContainerDied","Data":"ef437c5b6668718aaba07f77a5bb583d28dc3c5f1b9f2a41e66829415b554032"} Oct 30 12:51:35.149151 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.149097 2583 scope.go:122] "RemoveContainer" containerID="ef437c5b6668718aaba07f77a5bb583d28dc3c5f1b9f2a41e66829415b554032" Oct 30 12:51:35.152613 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.152591 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-bbb6d" event={"ID":"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462","Type":"ContainerStarted","Data":"aed13f0d4eafb4e57ebffd1a26d0b9a89d0ab5bb01759ca9a74c5a4860ad45b5"} Oct 30 12:51:35.154332 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.154306 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-d6rt6" event={"ID":"a5428d1a-8c8b-41ce-ad97-a729ec7a2e43","Type":"ContainerStarted","Data":"0c0a7e5307867f709ccc4a87d9633f02f62862e18e356e3011abb8924c16d1db"} Oct 30 12:51:35.260813 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.259649 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-bbb6d" podStartSLOduration=341.188799365 podStartE2EDuration="5m45.259629819s" totalImagesPullingTime="4.070830454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:45:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:35.258826633 +0000 UTC m=+35.981629603" watchObservedRunningTime="2026-10-30 12:51:35.259629819 +0000 UTC m=+35.982432790" Oct 30 12:51:35.378872 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.378842 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-l7z89"] Oct 30 12:51:35.399034 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.398992 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-vs88s-f66ls"] Oct 30 12:51:35.400797 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.400770 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:35.419640 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.419441 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 12:51:35.419640 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.419521 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 12:51:35.426302 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.421382 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-45hdr\"" Oct 30 12:51:35.426957 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.426614 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-l7z89"] Oct 30 12:51:35.426957 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.426678 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-vs88s-f66ls"] Oct 30 12:51:35.426957 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.426780 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:51:35.538964 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.538849 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-archives-path\") pod \"periodic-gathering-vs88s-f66ls\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:51:35.538964 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.538891 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5fjr\" (UniqueName: \"kubernetes.io/projected/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-kube-api-access-d5fjr\") pod \"periodic-gathering-vs88s-f66ls\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:51:35.538964 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.538918 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5cd3297b-e3fe-40e0-be87-88a67236d9f0-crio-socket\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:35.539184 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.539042 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cd3297b-e3fe-40e0-be87-88a67236d9f0-data-volume\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:35.539184 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.539151 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-trusted-ca-bundle\") pod \"periodic-gathering-vs88s-f66ls\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:51:35.539282 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.539247 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:35.539282 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.539273 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5cd3297b-e3fe-40e0-be87-88a67236d9f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:35.539373 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.539301 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhns5\" (UniqueName: \"kubernetes.io/projected/5cd3297b-e3fe-40e0-be87-88a67236d9f0-kube-api-access-fhns5\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:35.539427 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.539384 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-service-ca-bundle\") pod \"periodic-gathering-vs88s-f66ls\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:51:35.643104 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.640396 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:35.643104 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.640445 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5cd3297b-e3fe-40e0-be87-88a67236d9f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:35.643104 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.640477 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhns5\" (UniqueName: \"kubernetes.io/projected/5cd3297b-e3fe-40e0-be87-88a67236d9f0-kube-api-access-fhns5\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:35.643104 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.640554 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-service-ca-bundle\") pod \"periodic-gathering-vs88s-f66ls\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:51:35.643104 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.640592 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-archives-path\") pod \"periodic-gathering-vs88s-f66ls\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:51:35.643104 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.640618 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5fjr\" (UniqueName: \"kubernetes.io/projected/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-kube-api-access-d5fjr\") pod \"periodic-gathering-vs88s-f66ls\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:51:35.643104 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.640644 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5cd3297b-e3fe-40e0-be87-88a67236d9f0-crio-socket\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:35.643104 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.640686 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cd3297b-e3fe-40e0-be87-88a67236d9f0-data-volume\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:35.643104 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.640752 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-trusted-ca-bundle\") pod \"periodic-gathering-vs88s-f66ls\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:51:35.643104 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.641682 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-trusted-ca-bundle\") pod \"periodic-gathering-vs88s-f66ls\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:51:35.643104 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:35.641799 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 12:51:35.643104 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:35.641854 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls podName:5cd3297b-e3fe-40e0-be87-88a67236d9f0 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:36.14183633 +0000 UTC m=+36.864639293 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l7z89" (UID: "5cd3297b-e3fe-40e0-be87-88a67236d9f0") : secret "insights-runtime-extractor-tls" not found Oct 30 12:51:35.643104 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:35.641900 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 12:51:35.643104 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:35.641935 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5cd3297b-e3fe-40e0-be87-88a67236d9f0-kube-rbac-proxy-cm podName:5cd3297b-e3fe-40e0-be87-88a67236d9f0 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:36.141923181 +0000 UTC m=+36.864726129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5cd3297b-e3fe-40e0-be87-88a67236d9f0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l7z89" (UID: "5cd3297b-e3fe-40e0-be87-88a67236d9f0") : configmap "kube-rbac-proxy" not found Oct 30 12:51:35.643104 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.642641 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-service-ca-bundle\") pod \"periodic-gathering-vs88s-f66ls\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:51:35.643104 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.642975 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-archives-path\") pod \"periodic-gathering-vs88s-f66ls\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:51:35.644111 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.644075 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5cd3297b-e3fe-40e0-be87-88a67236d9f0-crio-socket\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:35.644383 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.644342 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cd3297b-e3fe-40e0-be87-88a67236d9f0-data-volume\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:35.669304 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.669276 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhns5\" (UniqueName: \"kubernetes.io/projected/5cd3297b-e3fe-40e0-be87-88a67236d9f0-kube-api-access-fhns5\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:35.670606 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.670550 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5fjr\" (UniqueName: \"kubernetes.io/projected/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-kube-api-access-d5fjr\") pod \"periodic-gathering-vs88s-f66ls\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:51:35.813144 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:35.812877 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:51:36.149341 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.149261 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:36.149341 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.149306 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5cd3297b-e3fe-40e0-be87-88a67236d9f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:36.149820 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:36.149460 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 12:51:36.149820 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:36.149528 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5cd3297b-e3fe-40e0-be87-88a67236d9f0-kube-rbac-proxy-cm podName:5cd3297b-e3fe-40e0-be87-88a67236d9f0 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:37.149508636 +0000 UTC m=+37.872311585 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5cd3297b-e3fe-40e0-be87-88a67236d9f0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l7z89" (UID: "5cd3297b-e3fe-40e0-be87-88a67236d9f0") : configmap "kube-rbac-proxy" not found Oct 30 12:51:36.149984 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:36.149960 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 12:51:36.150055 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:36.150025 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls podName:5cd3297b-e3fe-40e0-be87-88a67236d9f0 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:37.150009303 +0000 UTC m=+37.872812271 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l7z89" (UID: "5cd3297b-e3fe-40e0-be87-88a67236d9f0") : secret "insights-runtime-extractor-tls" not found Oct 30 12:51:36.173634 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.173520 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/1.log" Oct 30 12:51:36.176623 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.176528 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-vs88s-f66ls"] Oct 30 12:51:36.177096 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.177061 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/0.log" Oct 30 12:51:36.177193 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.177113 2583 generic.go:356] "Generic (PLEG): container finished" podID="fdddab43-acaf-4153-bb52-3e60044a35dc" containerID="72e9b907fdc4c8a3603e98b9c0298a772b043c460a2a2a72a020710cd8e9eeeb" exitCode=255 Oct 30 12:51:36.177374 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.177350 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" event={"ID":"fdddab43-acaf-4153-bb52-3e60044a35dc","Type":"ContainerDied","Data":"72e9b907fdc4c8a3603e98b9c0298a772b043c460a2a2a72a020710cd8e9eeeb"} Oct 30 12:51:36.177431 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.177401 2583 scope.go:122] "RemoveContainer" containerID="ef437c5b6668718aaba07f77a5bb583d28dc3c5f1b9f2a41e66829415b554032" Oct 30 12:51:36.177494 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.177453 2583 scope.go:122] "RemoveContainer" containerID="72e9b907fdc4c8a3603e98b9c0298a772b043c460a2a2a72a020710cd8e9eeeb" Oct 30 12:51:36.177676 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:36.177654 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-nbwj8_openshift-console-operator(fdddab43-acaf-4153-bb52-3e60044a35dc)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" podUID="fdddab43-acaf-4153-bb52-3e60044a35dc" Oct 30 12:51:36.179623 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.179259 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fnmsk" event={"ID":"d4ca5142-5b8a-4406-9ed0-06f9edea89bc","Type":"ContainerStarted","Data":"ae6800b8b2023abbdfe995882a49a6f38bdcd51d13fdc6551f9ce38811c6775a"} Oct 30 12:51:36.182378 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.182350 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp" event={"ID":"52ca5902-69f0-462f-ae35-cbbc7f378345","Type":"ContainerStarted","Data":"c633069f0be22f8934d108cfc759dcd9d430677376e758508912027f2386eb00"} Oct 30 12:51:36.184514 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.184488 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr" event={"ID":"9951ec02-ee4f-488c-8cab-3985569328f8","Type":"ContainerStarted","Data":"1a3e22b8be3970b1fec4a26737ca7d89090f863a588b21eafeea40eb71bbffcb"} Oct 30 12:51:36.214530 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:51:36.214499 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52a4a41e_8b25_4e5f_86ad_597dcd08cbd3.slice/crio-ae6e257ce29f7a6b8561fca5019d8978ad0f257f6e5e609b98d1c5d3eb418a16 WatchSource:0}: Error finding container ae6e257ce29f7a6b8561fca5019d8978ad0f257f6e5e609b98d1c5d3eb418a16: Status 404 returned error can't find the container with id ae6e257ce29f7a6b8561fca5019d8978ad0f257f6e5e609b98d1c5d3eb418a16 Oct 30 12:51:36.222881 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.222859 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-q68tl"] Oct 30 12:51:36.246110 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.246084 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-q68tl" Oct 30 12:51:36.250976 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.250950 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-pjfzl"] Oct 30 12:51:36.279677 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.279326 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-q68tl"] Oct 30 12:51:36.279677 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.279427 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-pjfzl" Oct 30 12:51:36.281796 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.281774 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-pjfzl"] Oct 30 12:51:36.294882 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.294837 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr" podStartSLOduration=341.663452297 podStartE2EDuration="5m46.294824174s" totalImagesPullingTime="4.631371877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:45:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:36.292863239 +0000 UTC m=+37.015666208" watchObservedRunningTime="2026-10-30 12:51:36.294824174 +0000 UTC m=+37.017627141" Oct 30 12:51:36.323314 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.323287 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-zxmvc\"" Oct 30 12:51:36.342488 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.342404 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp" podStartSLOduration=341.68870179 podStartE2EDuration="5m46.342384791s" totalImagesPullingTime="4.653683001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:45:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:36.340672236 +0000 UTC m=+37.063475205" watchObservedRunningTime="2026-10-30 12:51:36.342384791 +0000 UTC m=+37.065187761" Oct 30 12:51:36.349437 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.349397 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Oct 30 12:51:36.349729 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.349712 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Oct 30 12:51:36.352940 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.352917 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w928x\" (UniqueName: \"kubernetes.io/projected/6d7c203f-f696-471f-baf8-32c0ad647baf-kube-api-access-w928x\") pod \"migrator-66df5d496b-q68tl\" (UID: \"6d7c203f-f696-471f-baf8-32c0ad647baf\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-q68tl" Oct 30 12:51:36.448724 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.448625 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fnmsk" podStartSLOduration=2.475126562 podStartE2EDuration="7.448608278s" totalImagesPullingTime="4.973481716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:51:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:36.396232068 +0000 UTC m=+37.119035036" watchObservedRunningTime="2026-10-30 12:51:36.448608278 +0000 UTC m=+37.171411246" Oct 30 12:51:36.454634 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.454595 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cgzb\" (UniqueName: \"kubernetes.io/projected/85617466-6b8e-45d4-a45e-e9650078c0be-kube-api-access-9cgzb\") pod \"migrator-66df5d496b-pjfzl\" (UID: \"85617466-6b8e-45d4-a45e-e9650078c0be\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-pjfzl" Oct 30 12:51:36.454801 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.454655 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w928x\" (UniqueName: \"kubernetes.io/projected/6d7c203f-f696-471f-baf8-32c0ad647baf-kube-api-access-w928x\") pod \"migrator-66df5d496b-q68tl\" (UID: \"6d7c203f-f696-471f-baf8-32c0ad647baf\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-q68tl" Oct 30 12:51:36.493287 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.493254 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w928x\" (UniqueName: \"kubernetes.io/projected/6d7c203f-f696-471f-baf8-32c0ad647baf-kube-api-access-w928x\") pod \"migrator-66df5d496b-q68tl\" (UID: \"6d7c203f-f696-471f-baf8-32c0ad647baf\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-q68tl" Oct 30 12:51:36.555311 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.555272 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cgzb\" (UniqueName: \"kubernetes.io/projected/85617466-6b8e-45d4-a45e-e9650078c0be-kube-api-access-9cgzb\") pod \"migrator-66df5d496b-pjfzl\" (UID: \"85617466-6b8e-45d4-a45e-e9650078c0be\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-pjfzl" Oct 30 12:51:36.585173 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.585139 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cgzb\" (UniqueName: \"kubernetes.io/projected/85617466-6b8e-45d4-a45e-e9650078c0be-kube-api-access-9cgzb\") pod \"migrator-66df5d496b-pjfzl\" (UID: \"85617466-6b8e-45d4-a45e-e9650078c0be\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-pjfzl" Oct 30 12:51:36.592120 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.592095 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-q68tl" Oct 30 12:51:36.606450 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.606399 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-pjfzl" Oct 30 12:51:36.942554 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.942444 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-q68tl"] Oct 30 12:51:36.986763 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:36.986733 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-pjfzl"] Oct 30 12:51:36.990211 ip-10-0-141-147 kubenswrapper[2583]: W1030 12:51:36.990181 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85617466_6b8e_45d4_a45e_e9650078c0be.slice/crio-666dded619d9762c36a9a0c09fe3451d8392977966f7dc2aeecb6ccac98bbc49 WatchSource:0}: Error finding container 666dded619d9762c36a9a0c09fe3451d8392977966f7dc2aeecb6ccac98bbc49: Status 404 returned error can't find the container with id 666dded619d9762c36a9a0c09fe3451d8392977966f7dc2aeecb6ccac98bbc49 Oct 30 12:51:37.161201 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:37.161105 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:37.161609 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:37.161216 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5cd3297b-e3fe-40e0-be87-88a67236d9f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:37.161609 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:37.161290 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 12:51:37.161609 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:37.161343 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 12:51:37.161609 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:37.161379 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls podName:5cd3297b-e3fe-40e0-be87-88a67236d9f0 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:39.161356057 +0000 UTC m=+39.884159003 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l7z89" (UID: "5cd3297b-e3fe-40e0-be87-88a67236d9f0") : secret "insights-runtime-extractor-tls" not found Oct 30 12:51:37.161609 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:37.161401 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5cd3297b-e3fe-40e0-be87-88a67236d9f0-kube-rbac-proxy-cm podName:5cd3297b-e3fe-40e0-be87-88a67236d9f0 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:39.161391691 +0000 UTC m=+39.884194637 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5cd3297b-e3fe-40e0-be87-88a67236d9f0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l7z89" (UID: "5cd3297b-e3fe-40e0-be87-88a67236d9f0") : configmap "kube-rbac-proxy" not found Oct 30 12:51:37.190220 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:37.190189 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/1.log" Oct 30 12:51:37.190638 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:37.190618 2583 scope.go:122] "RemoveContainer" containerID="72e9b907fdc4c8a3603e98b9c0298a772b043c460a2a2a72a020710cd8e9eeeb" Oct 30 12:51:37.190849 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:37.190825 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-nbwj8_openshift-console-operator(fdddab43-acaf-4153-bb52-3e60044a35dc)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" podUID="fdddab43-acaf-4153-bb52-3e60044a35dc" Oct 30 12:51:37.192428 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:37.192395 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-q68tl" event={"ID":"6d7c203f-f696-471f-baf8-32c0ad647baf","Type":"ContainerStarted","Data":"6c71368cbc5b1b9f6817e4e8f210e1091f6386525d41a39c6d28f280459c8a9a"} Oct 30 12:51:37.193905 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:37.193879 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-pjfzl" event={"ID":"85617466-6b8e-45d4-a45e-e9650078c0be","Type":"ContainerStarted","Data":"666dded619d9762c36a9a0c09fe3451d8392977966f7dc2aeecb6ccac98bbc49"} Oct 30 12:51:37.196296 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:37.196269 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xsw6d" event={"ID":"0e13113b-b4d2-42d7-9e1c-642a790bcd55","Type":"ContainerStarted","Data":"cf6a92bd4d1d1407d0f68ac72ad11b3a8559f5d834abdd5249f6f19b5d8b46de"} Oct 30 12:51:37.199142 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:37.199113 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-vs88s-f66ls" event={"ID":"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3","Type":"ContainerStarted","Data":"0e5a8081a272ead021e67e3ae5d0138c0f8a93e8814ffec411f72e6bea8de29e"} Oct 30 12:51:37.199223 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:37.199160 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-vs88s-f66ls" event={"ID":"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3","Type":"ContainerStarted","Data":"ae6e257ce29f7a6b8561fca5019d8978ad0f257f6e5e609b98d1c5d3eb418a16"} Oct 30 12:51:37.291138 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:37.291020 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-vs88s-f66ls" podStartSLOduration=2.291001851 podStartE2EDuration="2.291001851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:51:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:37.289787311 +0000 UTC m=+38.012590279" watchObservedRunningTime="2026-10-30 12:51:37.291001851 +0000 UTC m=+38.013804820" Oct 30 12:51:37.336169 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:37.336097 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xsw6d" podStartSLOduration=340.126773916 podStartE2EDuration="5m46.336081776s" totalImagesPullingTime="6.20930786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:45:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:37.335000285 +0000 UTC m=+38.057803253" watchObservedRunningTime="2026-10-30 12:51:37.336081776 +0000 UTC m=+38.058884744" Oct 30 12:51:37.465487 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:37.465446 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wtgzn\" (UID: \"66992144-51dd-4c1d-95c8-63862bca97cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:37.465645 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:37.465535 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:37.465645 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:37.465618 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:37.465793 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:37.465776 2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 12:51:37.465857 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:37.465842 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls podName:f0d51a79-823b-4059-94c3-0535fc882a44 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:45.465824745 +0000 UTC m=+46.188627705 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6s9dp" (UID: "f0d51a79-823b-4059-94c3-0535fc882a44") : secret "cluster-monitoring-operator-tls" not found Oct 30 12:51:37.466227 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:37.466202 2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 12:51:37.466303 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:37.466264 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 12:51:37.466303 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:37.466282 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cfd489894-v5wlw: secret "image-registry-tls" not found Oct 30 12:51:37.466401 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:37.466269 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls podName:66992144-51dd-4c1d-95c8-63862bca97cc nodeName:}" failed. No retries permitted until 2026-10-30 12:51:45.466252896 +0000 UTC m=+46.189055857 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-wtgzn" (UID: "66992144-51dd-4c1d-95c8-63862bca97cc") : secret "samples-operator-tls" not found Oct 30 12:51:37.466401 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:37.466340 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls podName:fb047779-ad10-4f7e-a45a-4a154ab60ad8 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:45.466324043 +0000 UTC m=+46.189126994 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls") pod "image-registry-5cfd489894-v5wlw" (UID: "fb047779-ad10-4f7e-a45a-4a154ab60ad8") : secret "image-registry-tls" not found Oct 30 12:51:37.567426 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:37.566529 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:37.567426 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:37.566579 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-metrics-tls\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:37.567426 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:37.566633 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:37.567426 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:37.566662 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/109e0336-ecdb-4278-b5d3-72c7bdb80276-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-49h5k\" (UID: \"109e0336-ecdb-4278-b5d3-72c7bdb80276\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-49h5k" Oct 30 12:51:37.567426 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:37.566708 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e77ccc1-14a3-4f85-a214-854bd392733a-service-ca-bundle\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:37.567426 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:37.566924 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3e77ccc1-14a3-4f85-a214-854bd392733a-service-ca-bundle podName:3e77ccc1-14a3-4f85-a214-854bd392733a nodeName:}" failed. No retries permitted until 2026-10-30 12:51:45.566905582 +0000 UTC m=+46.289708531 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3e77ccc1-14a3-4f85-a214-854bd392733a-service-ca-bundle") pod "router-default-767cffcd47-qx67t" (UID: "3e77ccc1-14a3-4f85-a214-854bd392733a") : configmap references non-existent config key: service-ca.crt Oct 30 12:51:37.567426 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:37.567210 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 12:51:37.567426 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:37.567268 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs podName:3e77ccc1-14a3-4f85-a214-854bd392733a nodeName:}" failed. No retries permitted until 2026-10-30 12:51:45.567251383 +0000 UTC m=+46.290054352 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs") pod "router-default-767cffcd47-qx67t" (UID: "3e77ccc1-14a3-4f85-a214-854bd392733a") : secret "router-metrics-certs-default" not found Oct 30 12:51:37.567426 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:37.567277 2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 12:51:37.567426 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:37.567314 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/109e0336-ecdb-4278-b5d3-72c7bdb80276-networking-console-plugin-cert podName:109e0336-ecdb-4278-b5d3-72c7bdb80276 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:45.5673026 +0000 UTC m=+46.290105547 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/109e0336-ecdb-4278-b5d3-72c7bdb80276-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-49h5k" (UID: "109e0336-ecdb-4278-b5d3-72c7bdb80276") : secret "networking-console-plugin-cert" not found Oct 30 12:51:37.567426 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:37.567345 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 12:51:37.567426 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:37.567357 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cd77977d7-wrk42: secret "image-registry-tls" not found Oct 30 12:51:37.567426 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:37.567372 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 12:51:37.567426 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:37.567388 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls podName:5133b471-c5f8-4b35-b06c-1d8f429c3ba2 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:45.567378414 +0000 UTC m=+46.290181373 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls") pod "image-registry-5cd77977d7-wrk42" (UID: "5133b471-c5f8-4b35-b06c-1d8f429c3ba2") : secret "image-registry-tls" not found Oct 30 12:51:37.567426 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:37.567403 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-metrics-tls podName:38b6bfaf-2fb2-455b-9e7d-32586b3f216f nodeName:}" failed. No retries permitted until 2026-10-30 12:51:45.567393424 +0000 UTC m=+46.290196370 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-metrics-tls") pod "dns-default-cd6w9" (UID: "38b6bfaf-2fb2-455b-9e7d-32586b3f216f") : secret "dns-default-metrics-tls" not found Oct 30 12:51:38.206103 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:38.206010 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-d6rt6" event={"ID":"a5428d1a-8c8b-41ce-ad97-a729ec7a2e43","Type":"ContainerStarted","Data":"5f5dcdf77e4ce3be2b6d86153718244e7784da26fefedee3627c4879fed89974"} Oct 30 12:51:38.988098 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:38.988007 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-759xk_7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d/dns-node-resolver/0.log" Oct 30 12:51:39.191873 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:39.191835 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:39.191873 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:39.191880 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5cd3297b-e3fe-40e0-be87-88a67236d9f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:39.192149 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:39.192000 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 12:51:39.192149 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:39.192028 2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 12:51:39.192149 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:39.192098 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls podName:5cd3297b-e3fe-40e0-be87-88a67236d9f0 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:43.192056944 +0000 UTC m=+43.914859890 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l7z89" (UID: "5cd3297b-e3fe-40e0-be87-88a67236d9f0") : secret "insights-runtime-extractor-tls" not found Oct 30 12:51:39.192149 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:39.192122 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5cd3297b-e3fe-40e0-be87-88a67236d9f0-kube-rbac-proxy-cm podName:5cd3297b-e3fe-40e0-be87-88a67236d9f0 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:43.19211018 +0000 UTC m=+43.914913130 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5cd3297b-e3fe-40e0-be87-88a67236d9f0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l7z89" (UID: "5cd3297b-e3fe-40e0-be87-88a67236d9f0") : configmap "kube-rbac-proxy" not found Oct 30 12:51:39.211459 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:39.211427 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-q68tl" event={"ID":"6d7c203f-f696-471f-baf8-32c0ad647baf","Type":"ContainerStarted","Data":"5b68dcbe3f1db96954537ee0d6f34c9b04b8d078b09ff1a1fdd75b0e508c9df9"} Oct 30 12:51:39.213328 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:39.213299 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-pjfzl" event={"ID":"85617466-6b8e-45d4-a45e-e9650078c0be","Type":"ContainerStarted","Data":"0bf0e0d7c08030d2696ab8d5f9e59482609af8283bdf47bbafa9df6ef9e5a116"} Oct 30 12:51:39.213432 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:39.213399 2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:39.257155 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:39.257061 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-d6rt6" podStartSLOduration=35.808723259 podStartE2EDuration="39.257044791s" totalImagesPullingTime="3.448321532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:51:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:38.260609681 +0000 UTC m=+38.983412650" watchObservedRunningTime="2026-10-30 12:51:39.257044791 +0000 UTC m=+39.979847760" Oct 30 12:51:39.258287 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:39.258253 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-q68tl" podStartSLOduration=1.512640523 podStartE2EDuration="3.258244786s" totalImagesPullingTime="1.745604263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:51:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:39.25650775 +0000 UTC m=+39.979310712" watchObservedRunningTime="2026-10-30 12:51:39.258244786 +0000 UTC m=+39.981047755" Oct 30 12:51:39.303007 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:39.302959 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-pjfzl" podStartSLOduration=1.594591489 podStartE2EDuration="3.302946471s" totalImagesPullingTime="1.708354982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:51:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:51:39.301564881 +0000 UTC m=+40.024367852" watchObservedRunningTime="2026-10-30 12:51:39.302946471 +0000 UTC m=+40.025749482" Oct 30 12:51:39.964624 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:39.964582 2583 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:39.964789 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:39.964650 2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:39.965061 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:39.965045 2583 scope.go:122] "RemoveContainer" containerID="72e9b907fdc4c8a3603e98b9c0298a772b043c460a2a2a72a020710cd8e9eeeb" Oct 30 12:51:39.965305 ip-10-0-141-147 kubenswrapper[2583]: E1030 12:51:39.965280 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-nbwj8_openshift-console-operator(fdddab43-acaf-4153-bb52-3e60044a35dc)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" podUID="fdddab43-acaf-4153-bb52-3e60044a35dc" Oct 30 12:51:40.388746 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:40.388717 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vsf9w_2a12192e-f0c8-4354-b9fa-1b65b04ebc19/node-ca/0.log" Oct 30 12:51:40.788927 ip-10-0-141-147 kubenswrapper[2583]: I1030 12:51:40.788894 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fnmsk_d4ca5142-5b8a-4406-9ed0-06f9edea89bc/serve-healthcheck-canary/0.log" Oct 30 12:51:40.910250 ip-10-0-141-147 systemd[1]: Stopping Kubernetes Kubelet... Oct 30 12:51:40.921394 ip-10-0-141-147 systemd[1]: kubelet.service: Deactivated successfully. Oct 30 12:51:40.921563 ip-10-0-141-147 systemd[1]: Stopped Kubernetes Kubelet. Oct 30 12:51:40.921611 ip-10-0-141-147 systemd[1]: kubelet.service: Consumed 3.056s CPU time. Oct 30 12:51:40.932513 ip-10-0-141-147 systemd[1]: Starting Kubernetes Kubelet... Oct 30 12:51:41.002211 ip-10-0-141-147 kubenswrapper[5029]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 12:51:41.002211 ip-10-0-141-147 kubenswrapper[5029]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 12:51:41.002211 ip-10-0-141-147 kubenswrapper[5029]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 12:51:41.002211 ip-10-0-141-147 kubenswrapper[5029]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 12:51:41.004336 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004252 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 12:51:41.004336 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004334 5029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 12:51:41.004336 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004338 5029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 12:51:41.004444 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004342 5029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 12:51:41.004444 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004346 5029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 12:51:41.004444 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004349 5029 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 12:51:41.004444 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004352 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 12:51:41.004444 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004355 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 12:51:41.004444 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004358 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 12:51:41.004444 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004361 5029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 12:51:41.004444 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004364 5029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 12:51:41.004444 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004367 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 12:51:41.004444 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004369 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 12:51:41.004444 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004372 5029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 12:51:41.004444 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004375 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 12:51:41.004444 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004377 5029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 12:51:41.004444 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004380 5029 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 12:51:41.004444 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004383 5029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 12:51:41.004444 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004389 5029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 12:51:41.004444 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004392 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 12:51:41.004444 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004395 5029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 12:51:41.004444 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004398 5029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 12:51:41.004912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004401 5029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 12:51:41.004912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004404 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 12:51:41.004912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004406 5029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 12:51:41.004912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004409 5029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 12:51:41.004912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004411 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 12:51:41.004912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004414 5029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 12:51:41.004912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004417 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 12:51:41.004912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004419 5029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 12:51:41.004912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004422 5029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 12:51:41.004912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004425 5029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 12:51:41.004912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004428 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 12:51:41.004912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004430 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 12:51:41.004912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004433 5029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 12:51:41.004912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004435 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 12:51:41.004912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004438 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 12:51:41.004912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004441 5029 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 12:51:41.004912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004450 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 12:51:41.004912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004453 5029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 12:51:41.004912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004456 5029 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 12:51:41.004912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004458 5029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 12:51:41.005451 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004461 5029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 12:51:41.005451 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004464 5029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 12:51:41.005451 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004466 5029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 12:51:41.005451 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004469 5029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 12:51:41.005451 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004472 5029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 12:51:41.005451 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004474 5029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 12:51:41.005451 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004477 5029 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 12:51:41.005451 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004480 5029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 12:51:41.005451 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004483 5029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 12:51:41.005451 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004486 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 12:51:41.005451 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004490 5029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 12:51:41.005451 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004494 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 12:51:41.005451 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004497 5029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 12:51:41.005451 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004501 5029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 12:51:41.005451 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004503 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 12:51:41.005451 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004507 5029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 12:51:41.005451 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004510 5029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 12:51:41.005451 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004513 5029 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 12:51:41.005451 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004516 5029 feature_gate.go:386] unrecognized feature gate: Example Oct 30 12:51:41.005919 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004518 5029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 12:51:41.005919 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004521 5029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 12:51:41.005919 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004523 5029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 12:51:41.005919 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004526 5029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 12:51:41.005919 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004529 5029 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 12:51:41.005919 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004531 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 12:51:41.005919 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004534 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 12:51:41.005919 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004538 5029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 12:51:41.005919 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004542 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 12:51:41.005919 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004545 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 12:51:41.005919 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004549 5029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 12:51:41.005919 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004551 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 12:51:41.005919 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004554 5029 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 12:51:41.005919 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004557 5029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 12:51:41.005919 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004560 5029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 12:51:41.005919 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004562 5029 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 12:51:41.005919 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004565 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 12:51:41.005919 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004567 5029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 12:51:41.005919 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004570 5029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 12:51:41.006420 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004572 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 12:51:41.006420 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004575 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 12:51:41.006420 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004578 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 12:51:41.006420 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004581 5029 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 12:51:41.006420 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004584 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 12:51:41.006420 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004587 5029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 12:51:41.006420 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004590 5029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 12:51:41.006420 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004593 5029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 12:51:41.006420 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004595 5029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 12:51:41.006420 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004598 5029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 12:51:41.006420 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004601 5029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 12:51:41.006420 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004603 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 12:51:41.006420 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004606 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 12:51:41.006420 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004608 5029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 12:51:41.006420 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004611 5029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 12:51:41.006420 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004613 5029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 12:51:41.006420 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004616 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 12:51:41.006420 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004619 5029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 12:51:41.006420 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004621 5029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 12:51:41.006897 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004624 5029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 12:51:41.006897 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004626 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 12:51:41.006897 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004629 5029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 12:51:41.006897 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004631 5029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 12:51:41.006897 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004634 5029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 12:51:41.006897 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004637 5029 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 12:51:41.006897 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.004640 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 12:51:41.006897 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005457 5029 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 12:51:41.006897 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005466 5029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 12:51:41.006897 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005470 5029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 12:51:41.006897 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005473 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 12:51:41.006897 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005476 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 12:51:41.006897 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005479 5029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 12:51:41.006897 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005481 5029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 12:51:41.006897 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005484 5029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 12:51:41.006897 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005486 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 12:51:41.006897 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005490 5029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 12:51:41.006897 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005492 5029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 12:51:41.006897 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005495 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 12:51:41.006897 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005498 5029 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 12:51:41.007443 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005500 5029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 12:51:41.007443 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005503 5029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 12:51:41.007443 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005506 5029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 12:51:41.007443 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005509 5029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 12:51:41.007443 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005512 5029 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 12:51:41.007443 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005514 5029 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 12:51:41.007443 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005517 5029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 12:51:41.007443 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005519 5029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 12:51:41.007443 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005523 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 12:51:41.007443 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005525 5029 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 12:51:41.007443 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005528 5029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 12:51:41.007443 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005530 5029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 12:51:41.007443 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005533 5029 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 12:51:41.007443 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005535 5029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 12:51:41.007443 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005538 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 12:51:41.007443 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005541 5029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 12:51:41.007443 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005544 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 12:51:41.007443 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005546 5029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 12:51:41.007443 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005549 5029 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 12:51:41.007443 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005552 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 12:51:41.008193 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005554 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 12:51:41.008193 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005557 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 12:51:41.008193 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005559 5029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 12:51:41.008193 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005562 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 12:51:41.008193 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005564 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 12:51:41.008193 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005567 5029 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 12:51:41.008193 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005570 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 12:51:41.008193 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005572 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 12:51:41.008193 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005575 5029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 12:51:41.008193 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005578 5029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 12:51:41.008193 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005581 5029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 12:51:41.008193 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005583 5029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 12:51:41.008193 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005586 5029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 12:51:41.008193 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005588 5029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 12:51:41.008193 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005591 5029 feature_gate.go:386] unrecognized feature gate: Example Oct 30 12:51:41.008193 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005594 5029 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 12:51:41.008193 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005597 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 12:51:41.008193 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005600 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 12:51:41.008193 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005602 5029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 12:51:41.008193 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005606 5029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 12:51:41.008927 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005610 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 12:51:41.008927 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005613 5029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 12:51:41.008927 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005617 5029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 12:51:41.008927 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005621 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 12:51:41.008927 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005625 5029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 12:51:41.008927 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005629 5029 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 12:51:41.008927 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005632 5029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 12:51:41.008927 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005634 5029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 12:51:41.008927 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005637 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 12:51:41.008927 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005640 5029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 12:51:41.008927 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005642 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 12:51:41.008927 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005645 5029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 12:51:41.008927 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005648 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 12:51:41.008927 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005650 5029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 12:51:41.008927 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005653 5029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 12:51:41.008927 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005656 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 12:51:41.008927 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005658 5029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 12:51:41.008927 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005661 5029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 12:51:41.008927 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005664 5029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 12:51:41.009433 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005666 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 12:51:41.009433 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005670 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 12:51:41.009433 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005672 5029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 12:51:41.009433 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005675 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 12:51:41.009433 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005678 5029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 12:51:41.009433 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005680 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 12:51:41.009433 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005683 5029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 12:51:41.009433 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005686 5029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 12:51:41.009433 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005688 5029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 12:51:41.009433 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005694 5029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 12:51:41.009433 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005697 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 12:51:41.009433 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005699 5029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 12:51:41.009433 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005702 5029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 12:51:41.009433 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005705 5029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 12:51:41.009433 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005707 5029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 12:51:41.009433 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005710 5029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 12:51:41.009433 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005713 5029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 12:51:41.009433 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005716 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 12:51:41.009433 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005719 5029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 12:51:41.009910 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005722 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 12:51:41.009910 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005724 5029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 12:51:41.009910 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005727 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 12:51:41.009910 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005730 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 12:51:41.009910 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005733 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 12:51:41.009910 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005735 5029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 12:51:41.009910 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005739 5029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 12:51:41.009910 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005741 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 12:51:41.009910 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005744 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 12:51:41.009910 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005746 5029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 12:51:41.009910 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005749 5029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 12:51:41.009910 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005752 5029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 12:51:41.009910 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005754 5029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 12:51:41.009910 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005757 5029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 12:51:41.009910 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.005759 5029 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 12:51:41.009910 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006278 5029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 12:51:41.009910 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006285 5029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 12:51:41.009910 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006288 5029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 12:51:41.009910 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006292 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 12:51:41.010437 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006295 5029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 12:51:41.010437 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006299 5029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 12:51:41.010437 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006301 5029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 12:51:41.010437 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006310 5029 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 12:51:41.010437 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006313 5029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 12:51:41.010437 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006316 5029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 12:51:41.010437 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006319 5029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 12:51:41.010437 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006322 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 12:51:41.010437 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006324 5029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 12:51:41.010437 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006327 5029 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 12:51:41.010437 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006330 5029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 12:51:41.010437 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006332 5029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 12:51:41.010437 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006335 5029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 12:51:41.010437 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006337 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 12:51:41.010437 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006341 5029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 12:51:41.010437 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006346 5029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 12:51:41.010437 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006349 5029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 12:51:41.010437 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006351 5029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 12:51:41.010437 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006354 5029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 12:51:41.010926 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006357 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 12:51:41.010926 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006360 5029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 12:51:41.010926 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006362 5029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 12:51:41.010926 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006365 5029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 12:51:41.010926 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006367 5029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 12:51:41.010926 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006370 5029 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 12:51:41.010926 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006373 5029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 12:51:41.010926 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006375 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 12:51:41.010926 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006378 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 12:51:41.010926 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006380 5029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 12:51:41.010926 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006383 5029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 12:51:41.010926 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006385 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 12:51:41.010926 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006388 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 12:51:41.010926 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006390 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 12:51:41.010926 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006393 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 12:51:41.010926 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006395 5029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 12:51:41.010926 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006403 5029 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 12:51:41.010926 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006406 5029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 12:51:41.010926 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006409 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 12:51:41.011401 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006412 5029 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 12:51:41.011401 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006414 5029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 12:51:41.011401 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006417 5029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 12:51:41.011401 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006419 5029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 12:51:41.011401 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006422 5029 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 12:51:41.011401 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006424 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 12:51:41.011401 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006427 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 12:51:41.011401 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006430 5029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 12:51:41.011401 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006433 5029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 12:51:41.011401 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006435 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 12:51:41.011401 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006438 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 12:51:41.011401 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006440 5029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 12:51:41.011401 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006443 5029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 12:51:41.011401 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006446 5029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 12:51:41.011401 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006449 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 12:51:41.011401 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006452 5029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 12:51:41.011401 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006454 5029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 12:51:41.011401 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006457 5029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 12:51:41.011401 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006459 5029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 12:51:41.011401 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006462 5029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 12:51:41.012159 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006465 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 12:51:41.012159 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006468 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 12:51:41.012159 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006470 5029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 12:51:41.012159 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006472 5029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 12:51:41.012159 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006475 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 12:51:41.012159 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006478 5029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 12:51:41.012159 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006480 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 12:51:41.012159 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006483 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 12:51:41.012159 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006485 5029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 12:51:41.012159 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006490 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 12:51:41.012159 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006492 5029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 12:51:41.012159 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006495 5029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 12:51:41.012159 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006498 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 12:51:41.012159 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006500 5029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 12:51:41.012159 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006503 5029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 12:51:41.012159 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006506 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 12:51:41.012159 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006509 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 12:51:41.012159 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006511 5029 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 12:51:41.012159 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006514 5029 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 12:51:41.012159 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006516 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 12:51:41.013037 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006519 5029 feature_gate.go:386] unrecognized feature gate: Example Oct 30 12:51:41.013037 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006522 5029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 12:51:41.013037 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006524 5029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 12:51:41.013037 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006527 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 12:51:41.013037 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006529 5029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 12:51:41.013037 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006532 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 12:51:41.013037 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006534 5029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 12:51:41.013037 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006537 5029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 12:51:41.013037 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006540 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 12:51:41.013037 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006543 5029 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 12:51:41.013037 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006545 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 12:51:41.013037 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006548 5029 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 12:51:41.013037 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006550 5029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 12:51:41.013037 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006553 5029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 12:51:41.013037 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006555 5029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 12:51:41.013037 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006558 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 12:51:41.013037 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006560 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 12:51:41.013037 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006563 5029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 12:51:41.013037 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006565 5029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 12:51:41.013037 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006568 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006573 5029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006577 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006580 5029 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.006583 5029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006687 5029 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006698 5029 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006707 5029 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006712 5029 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006716 5029 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006720 5029 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006725 5029 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006729 5029 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006732 5029 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006735 5029 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006739 5029 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006742 5029 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006745 5029 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006748 5029 flags.go:64] FLAG: --cgroup-root="" Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006751 5029 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006754 5029 flags.go:64] FLAG: --client-ca-file="" Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006758 5029 flags.go:64] FLAG: --cloud-provider="external" Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006761 5029 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 12:51:41.013912 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006764 5029 flags.go:64] FLAG: --cluster-domain="" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006767 5029 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006770 5029 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006773 5029 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006777 5029 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006781 5029 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006784 5029 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006787 5029 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006790 5029 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006793 5029 flags.go:64] FLAG: --contention-profiling="false" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006797 5029 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006801 5029 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006805 5029 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006808 5029 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006812 5029 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006815 5029 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006818 5029 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006821 5029 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006824 5029 flags.go:64] FLAG: --enable-server="true" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006827 5029 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006831 5029 flags.go:64] FLAG: --event-burst="100" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006834 5029 flags.go:64] FLAG: --event-qps="50" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006837 5029 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006841 5029 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 12:51:41.014934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006844 5029 flags.go:64] FLAG: --eviction-hard="" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006848 5029 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006852 5029 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006855 5029 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006858 5029 flags.go:64] FLAG: --eviction-soft="" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006861 5029 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006864 5029 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006870 5029 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006873 5029 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006876 5029 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006879 5029 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006882 5029 flags.go:64] FLAG: --feature-gates="" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006886 5029 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006889 5029 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006892 5029 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006895 5029 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006898 5029 flags.go:64] FLAG: --healthz-port="10248" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006902 5029 flags.go:64] FLAG: --help="false" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006905 5029 flags.go:64] FLAG: --hostname-override="ip-10-0-141-147.ec2.internal" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006910 5029 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006913 5029 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006916 5029 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006919 5029 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 12:51:41.016026 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006923 5029 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006926 5029 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006929 5029 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006931 5029 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006934 5029 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006937 5029 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006940 5029 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006943 5029 flags.go:64] FLAG: --kube-reserved="" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006946 5029 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006949 5029 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006953 5029 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006956 5029 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006959 5029 flags.go:64] FLAG: --lock-file="" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006962 5029 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006965 5029 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006969 5029 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006974 5029 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006977 5029 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006980 5029 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006983 5029 flags.go:64] FLAG: --logging-format="text" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006987 5029 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006990 5029 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006993 5029 flags.go:64] FLAG: --manifest-url="" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.006996 5029 flags.go:64] FLAG: --manifest-url-header="" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007001 5029 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 12:51:41.017139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007004 5029 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007008 5029 flags.go:64] FLAG: --max-pods="110" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007011 5029 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007016 5029 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007019 5029 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007022 5029 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007025 5029 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007028 5029 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007031 5029 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007038 5029 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007041 5029 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007044 5029 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007048 5029 flags.go:64] FLAG: --pod-cidr="" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007051 5029 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007054 5029 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007057 5029 flags.go:64] FLAG: --pods-per-core="0" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007060 5029 flags.go:64] FLAG: --port="10250" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007079 5029 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007085 5029 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08310a1314a7c0a15" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007088 5029 flags.go:64] FLAG: --qos-reserved="" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007091 5029 flags.go:64] FLAG: --read-only-port="10255" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007095 5029 flags.go:64] FLAG: --register-node="true" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007098 5029 flags.go:64] FLAG: --register-with-taints="" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007102 5029 flags.go:64] FLAG: --registry-burst="10" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007105 5029 flags.go:64] FLAG: --registry-qps="5" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007109 5029 flags.go:64] FLAG: --reserved-cpus="" Oct 30 12:51:41.018241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007112 5029 flags.go:64] FLAG: --reserved-memory="" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007116 5029 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007120 5029 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007123 5029 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007126 5029 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007129 5029 flags.go:64] FLAG: --runonce="false" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007132 5029 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007135 5029 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007138 5029 flags.go:64] FLAG: --seccomp-default="false" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007142 5029 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007145 5029 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007148 5029 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007151 5029 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007154 5029 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007157 5029 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007160 5029 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007163 5029 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007166 5029 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007169 5029 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007172 5029 flags.go:64] FLAG: --system-cgroups="" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007175 5029 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007181 5029 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007184 5029 flags.go:64] FLAG: --tls-cert-file="" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007188 5029 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007191 5029 flags.go:64] FLAG: --tls-min-version="" Oct 30 12:51:41.019650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007194 5029 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 12:51:41.020784 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007197 5029 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 12:51:41.020784 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007200 5029 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 12:51:41.020784 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007203 5029 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 12:51:41.020784 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007206 5029 flags.go:64] FLAG: --v="2" Oct 30 12:51:41.020784 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007210 5029 flags.go:64] FLAG: --version="false" Oct 30 12:51:41.020784 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007214 5029 flags.go:64] FLAG: --vmodule="" Oct 30 12:51:41.020784 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007219 5029 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 12:51:41.020784 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.007223 5029 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 12:51:41.020784 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008515 5029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 12:51:41.020784 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008589 5029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 12:51:41.020784 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008612 5029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 12:51:41.020784 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008620 5029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 12:51:41.020784 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008626 5029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 12:51:41.020784 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008631 5029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 12:51:41.020784 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008635 5029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 12:51:41.020784 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008642 5029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 12:51:41.020784 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008646 5029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 12:51:41.020784 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008651 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 12:51:41.020784 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008655 5029 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 12:51:41.020784 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008660 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 12:51:41.020784 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008664 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 12:51:41.020784 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008674 5029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 12:51:41.021540 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008679 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 12:51:41.021540 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008685 5029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 12:51:41.021540 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008689 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 12:51:41.021540 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008694 5029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 12:51:41.021540 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008699 5029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 12:51:41.021540 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008704 5029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 12:51:41.021540 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008752 5029 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 12:51:41.021540 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008879 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 12:51:41.021540 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008884 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 12:51:41.021540 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008887 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 12:51:41.021540 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008890 5029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 12:51:41.021540 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008893 5029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 12:51:41.021540 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008897 5029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 12:51:41.021540 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008900 5029 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 12:51:41.021540 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008903 5029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 12:51:41.021540 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008908 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 12:51:41.021540 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008911 5029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 12:51:41.021540 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008914 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 12:51:41.021540 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008917 5029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 12:51:41.022164 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008920 5029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 12:51:41.022164 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008922 5029 feature_gate.go:386] unrecognized feature gate: Example Oct 30 12:51:41.022164 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008925 5029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 12:51:41.022164 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008928 5029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 12:51:41.022164 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008930 5029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 12:51:41.022164 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008933 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 12:51:41.022164 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008936 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 12:51:41.022164 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008939 5029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 12:51:41.022164 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008942 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 12:51:41.022164 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008945 5029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 12:51:41.022164 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008948 5029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 12:51:41.022164 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008950 5029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 12:51:41.022164 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008954 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 12:51:41.022164 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008957 5029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 12:51:41.022164 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008959 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 12:51:41.022164 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008962 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 12:51:41.022164 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008965 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 12:51:41.022164 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008968 5029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 12:51:41.022164 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008971 5029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 12:51:41.022164 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008974 5029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 12:51:41.022662 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008977 5029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 12:51:41.022662 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008980 5029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 12:51:41.022662 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008983 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 12:51:41.022662 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008987 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 12:51:41.022662 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008990 5029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 12:51:41.022662 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008993 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 12:51:41.022662 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008996 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 12:51:41.022662 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.008998 5029 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 12:51:41.022662 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009001 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 12:51:41.022662 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009003 5029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 12:51:41.022662 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009006 5029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 12:51:41.022662 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009009 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 12:51:41.022662 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009011 5029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 12:51:41.022662 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009014 5029 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 12:51:41.022662 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009019 5029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 12:51:41.022662 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009024 5029 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 12:51:41.022662 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009027 5029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 12:51:41.022662 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009030 5029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 12:51:41.022662 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009035 5029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 12:51:41.022662 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009038 5029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 12:51:41.023284 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009040 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 12:51:41.023284 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009043 5029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 12:51:41.023284 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009046 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 12:51:41.023284 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009049 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 12:51:41.023284 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009052 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 12:51:41.023284 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009054 5029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 12:51:41.023284 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009057 5029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 12:51:41.023284 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009060 5029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 12:51:41.023284 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009077 5029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 12:51:41.023284 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009082 5029 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 12:51:41.023284 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009087 5029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 12:51:41.023284 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009090 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 12:51:41.023284 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009093 5029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 12:51:41.023284 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009106 5029 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 12:51:41.023284 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009111 5029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 12:51:41.023284 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009116 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 12:51:41.023284 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009121 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 12:51:41.023284 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009125 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 12:51:41.023284 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009130 5029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 12:51:41.023754 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009133 5029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 12:51:41.023754 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009136 5029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 12:51:41.023754 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009138 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 12:51:41.023754 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009141 5029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 12:51:41.023754 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009144 5029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 12:51:41.023754 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009147 5029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 12:51:41.023754 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009150 5029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 12:51:41.023754 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009152 5029 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 12:51:41.023754 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009155 5029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 12:51:41.023754 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009158 5029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 12:51:41.023754 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009161 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 12:51:41.023754 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009164 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 12:51:41.023754 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009166 5029 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 12:51:41.023754 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009169 5029 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 12:51:41.023754 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.009204 5029 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 12:51:41.023754 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009254 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 12:51:41.023754 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009260 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 12:51:41.024228 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009264 5029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 12:51:41.024228 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009268 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 12:51:41.024228 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009271 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 12:51:41.024228 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009275 5029 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 12:51:41.024228 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009278 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 12:51:41.024228 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009280 5029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 12:51:41.024228 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009283 5029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 12:51:41.024228 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009286 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 12:51:41.024228 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009289 5029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 12:51:41.024228 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009292 5029 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 12:51:41.024228 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009296 5029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 12:51:41.024228 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009300 5029 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 12:51:41.024228 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009303 5029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 12:51:41.024228 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009305 5029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 12:51:41.024228 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009308 5029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 12:51:41.024228 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009310 5029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 12:51:41.024228 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009313 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 12:51:41.024228 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009316 5029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 12:51:41.024228 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009318 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 12:51:41.024832 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009322 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 12:51:41.024832 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009324 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 12:51:41.024832 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009327 5029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 12:51:41.024832 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009330 5029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 12:51:41.024832 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009332 5029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 12:51:41.024832 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009335 5029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 12:51:41.024832 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009338 5029 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 12:51:41.024832 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009340 5029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 12:51:41.024832 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009343 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 12:51:41.024832 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009345 5029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 12:51:41.024832 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009348 5029 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 12:51:41.024832 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009354 5029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 12:51:41.024832 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009358 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 12:51:41.024832 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009361 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 12:51:41.024832 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009364 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 12:51:41.024832 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009367 5029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 12:51:41.024832 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009369 5029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 12:51:41.024832 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009372 5029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 12:51:41.024832 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009375 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 12:51:41.025346 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009378 5029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 12:51:41.025346 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009380 5029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 12:51:41.025346 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009383 5029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 12:51:41.025346 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009386 5029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 12:51:41.025346 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009388 5029 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 12:51:41.025346 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009391 5029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 12:51:41.025346 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009394 5029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 12:51:41.025346 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009396 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 12:51:41.025346 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009399 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 12:51:41.025346 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009402 5029 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 12:51:41.025346 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009405 5029 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 12:51:41.025346 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009408 5029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 12:51:41.025346 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009410 5029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 12:51:41.025346 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009413 5029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 12:51:41.025346 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009416 5029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 12:51:41.025346 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009419 5029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 12:51:41.025346 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009422 5029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 12:51:41.025346 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009424 5029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 12:51:41.025346 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009427 5029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 12:51:41.025346 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009430 5029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 12:51:41.025848 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009433 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 12:51:41.025848 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009435 5029 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 12:51:41.025848 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009438 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 12:51:41.025848 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009441 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 12:51:41.025848 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009444 5029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 12:51:41.025848 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009447 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 12:51:41.025848 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009450 5029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 12:51:41.025848 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009453 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 12:51:41.025848 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009456 5029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 12:51:41.025848 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009460 5029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 12:51:41.025848 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009462 5029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 12:51:41.025848 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009465 5029 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 12:51:41.025848 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009467 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 12:51:41.025848 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009470 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 12:51:41.025848 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009473 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 12:51:41.025848 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009482 5029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 12:51:41.025848 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009485 5029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 12:51:41.025848 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009488 5029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 12:51:41.025848 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009491 5029 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 12:51:41.026341 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009494 5029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 12:51:41.026341 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009496 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 12:51:41.026341 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009499 5029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 12:51:41.026341 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009501 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 12:51:41.026341 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009504 5029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 12:51:41.026341 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009507 5029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 12:51:41.026341 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009509 5029 feature_gate.go:386] unrecognized feature gate: Example Oct 30 12:51:41.026341 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009512 5029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 12:51:41.026341 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009514 5029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 12:51:41.026341 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009517 5029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 12:51:41.026341 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009520 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 12:51:41.026341 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009522 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 12:51:41.026341 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009525 5029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 12:51:41.026341 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009528 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 12:51:41.026341 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009531 5029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 12:51:41.026341 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009534 5029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 12:51:41.026341 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009536 5029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 12:51:41.026341 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009539 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 12:51:41.026341 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009542 5029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 12:51:41.026840 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009545 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 12:51:41.026840 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009547 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 12:51:41.026840 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009550 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 12:51:41.026840 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009554 5029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 12:51:41.026840 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009556 5029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 12:51:41.026840 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009559 5029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 12:51:41.026840 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009562 5029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 12:51:41.026840 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.009564 5029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 12:51:41.026840 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.014228 5029 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 12:51:41.026840 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.014249 5029 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 12:51:41.026840 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.014274 5029 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 12:51:41.026840 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.014282 5029 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 12:51:41.026840 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014367 5029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 12:51:41.026840 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014374 5029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 12:51:41.026840 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014380 5029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 12:51:41.026840 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014384 5029 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 12:51:41.026840 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014391 5029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 12:51:41.026840 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014399 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 12:51:41.026840 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014404 5029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 12:51:41.027472 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014408 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 12:51:41.027472 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014413 5029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 12:51:41.027472 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014417 5029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 12:51:41.027472 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014422 5029 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 12:51:41.027472 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014426 5029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 12:51:41.027472 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014430 5029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 12:51:41.027472 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014435 5029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 12:51:41.027472 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014439 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 12:51:41.027472 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014444 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 12:51:41.027472 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014448 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 12:51:41.027472 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014453 5029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 12:51:41.027472 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014458 5029 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 12:51:41.027472 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014462 5029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 12:51:41.027472 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014467 5029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 12:51:41.027472 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014472 5029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 12:51:41.027472 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014477 5029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 12:51:41.027472 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014482 5029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 12:51:41.027472 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014486 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 12:51:41.027472 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014490 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 12:51:41.028248 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014494 5029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 12:51:41.028248 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014498 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 12:51:41.028248 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014503 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 12:51:41.028248 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014509 5029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 12:51:41.028248 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014513 5029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 12:51:41.028248 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014518 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 12:51:41.028248 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014522 5029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 12:51:41.028248 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014526 5029 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 12:51:41.028248 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014531 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 12:51:41.028248 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014535 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 12:51:41.028248 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014539 5029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 12:51:41.028248 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014543 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 12:51:41.028248 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014548 5029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 12:51:41.028248 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014552 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 12:51:41.028248 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014556 5029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 12:51:41.028248 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014561 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 12:51:41.028248 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014566 5029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 12:51:41.028248 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014570 5029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 12:51:41.028248 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014575 5029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 12:51:41.028248 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014579 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 12:51:41.028949 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014584 5029 feature_gate.go:386] unrecognized feature gate: Example Oct 30 12:51:41.028949 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014588 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 12:51:41.028949 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014592 5029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 12:51:41.028949 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014597 5029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 12:51:41.028949 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014601 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 12:51:41.028949 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014606 5029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 12:51:41.028949 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014610 5029 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 12:51:41.028949 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014614 5029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 12:51:41.028949 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014619 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 12:51:41.028949 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014623 5029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 12:51:41.028949 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014627 5029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 12:51:41.028949 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014631 5029 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 12:51:41.028949 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014636 5029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 12:51:41.028949 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014640 5029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 12:51:41.028949 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014645 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 12:51:41.028949 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014649 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 12:51:41.028949 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014655 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 12:51:41.028949 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014660 5029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 12:51:41.028949 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014664 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 12:51:41.029456 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014669 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 12:51:41.029456 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014674 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 12:51:41.029456 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014678 5029 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 12:51:41.029456 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014682 5029 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 12:51:41.029456 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014686 5029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 12:51:41.029456 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014690 5029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 12:51:41.029456 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014695 5029 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 12:51:41.029456 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014699 5029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 12:51:41.029456 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014703 5029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 12:51:41.029456 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014707 5029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 12:51:41.029456 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014711 5029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 12:51:41.029456 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014716 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 12:51:41.029456 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014720 5029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 12:51:41.029456 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014724 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 12:51:41.029456 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014728 5029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 12:51:41.029456 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014733 5029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 12:51:41.029456 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014738 5029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 12:51:41.029456 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014742 5029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 12:51:41.029456 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014746 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 12:51:41.029456 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014751 5029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 12:51:41.030136 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014755 5029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 12:51:41.030136 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014760 5029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 12:51:41.030136 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014764 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 12:51:41.030136 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014768 5029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 12:51:41.030136 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014772 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 12:51:41.030136 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014776 5029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 12:51:41.030136 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014781 5029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 12:51:41.030136 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014787 5029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 12:51:41.030136 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014793 5029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 12:51:41.030136 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014798 5029 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 12:51:41.030136 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014804 5029 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 12:51:41.030136 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014808 5029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 12:51:41.030136 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014814 5029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 12:51:41.030136 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014818 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 12:51:41.030136 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014823 5029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 12:51:41.030136 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014827 5029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 12:51:41.030136 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014832 5029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 12:51:41.030136 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014837 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 12:51:41.030136 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014841 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 12:51:41.030136 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014845 5029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 12:51:41.030808 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.014849 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 12:51:41.030808 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.014891 5029 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 12:51:41.030808 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015146 5029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 12:51:41.030808 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015157 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 12:51:41.030808 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015163 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 12:51:41.030808 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015168 5029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 12:51:41.030808 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015173 5029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 12:51:41.030808 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015178 5029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 12:51:41.030808 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015183 5029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 12:51:41.030808 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015187 5029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 12:51:41.030808 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015191 5029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 12:51:41.030808 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015196 5029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 12:51:41.030808 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015200 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 12:51:41.030808 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015205 5029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 12:51:41.030808 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015210 5029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 12:51:41.031261 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015214 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 12:51:41.031261 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015218 5029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 12:51:41.031261 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015222 5029 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 12:51:41.031261 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015227 5029 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 12:51:41.031261 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015231 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 12:51:41.031261 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015235 5029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 12:51:41.031261 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015239 5029 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 12:51:41.031261 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015244 5029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 12:51:41.031261 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015249 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 12:51:41.031261 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015254 5029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 12:51:41.031261 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015259 5029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 12:51:41.031261 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015263 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 12:51:41.031261 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015267 5029 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 12:51:41.031261 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015271 5029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 12:51:41.031261 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015275 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 12:51:41.031261 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015279 5029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 12:51:41.031261 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015289 5029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 12:51:41.031261 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015293 5029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 12:51:41.031261 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015298 5029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 12:51:41.031261 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015303 5029 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 12:51:41.031743 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015307 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 12:51:41.031743 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015312 5029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 12:51:41.031743 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015316 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 12:51:41.031743 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015321 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 12:51:41.031743 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015325 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 12:51:41.031743 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015330 5029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 12:51:41.031743 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015335 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 12:51:41.031743 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015339 5029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 12:51:41.031743 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015344 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 12:51:41.031743 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015348 5029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 12:51:41.031743 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015352 5029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 12:51:41.031743 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015356 5029 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 12:51:41.031743 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015360 5029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 12:51:41.031743 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015365 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 12:51:41.031743 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015369 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 12:51:41.031743 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015374 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 12:51:41.031743 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015379 5029 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 12:51:41.031743 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015383 5029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 12:51:41.031743 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015388 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 12:51:41.031743 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015392 5029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 12:51:41.032332 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015396 5029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 12:51:41.032332 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015402 5029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 12:51:41.032332 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015406 5029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 12:51:41.032332 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015410 5029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 12:51:41.032332 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015415 5029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 12:51:41.032332 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015419 5029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 12:51:41.032332 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015424 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 12:51:41.032332 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015428 5029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 12:51:41.032332 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015435 5029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 12:51:41.032332 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015440 5029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 12:51:41.032332 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015444 5029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 12:51:41.032332 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015448 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 12:51:41.032332 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015452 5029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 12:51:41.032332 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015457 5029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 12:51:41.032332 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015462 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 12:51:41.032332 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015466 5029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 12:51:41.032332 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015471 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 12:51:41.032332 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015475 5029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 12:51:41.032332 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015480 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 12:51:41.032332 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015484 5029 feature_gate.go:386] unrecognized feature gate: Example Oct 30 12:51:41.032828 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015489 5029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 12:51:41.032828 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015494 5029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 12:51:41.032828 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015498 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 12:51:41.032828 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015502 5029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 12:51:41.032828 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015509 5029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 12:51:41.032828 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015516 5029 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 12:51:41.032828 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015521 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 12:51:41.032828 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015525 5029 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 12:51:41.032828 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015530 5029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 12:51:41.032828 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015535 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 12:51:41.032828 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015542 5029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 12:51:41.032828 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015548 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 12:51:41.032828 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015552 5029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 12:51:41.032828 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015557 5029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 12:51:41.032828 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015562 5029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 12:51:41.032828 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015567 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 12:51:41.032828 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015571 5029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 12:51:41.032828 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015576 5029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 12:51:41.032828 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015580 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 12:51:41.033314 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015584 5029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 12:51:41.033314 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015591 5029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 12:51:41.033314 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015595 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 12:51:41.033314 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015599 5029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 12:51:41.033314 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015604 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 12:51:41.033314 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015608 5029 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 12:51:41.033314 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015613 5029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 12:51:41.033314 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015618 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 12:51:41.033314 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015622 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 12:51:41.033314 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015626 5029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 12:51:41.033314 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015631 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 12:51:41.033314 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015635 5029 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 12:51:41.033314 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015640 5029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 12:51:41.033314 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015644 5029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 12:51:41.033314 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.015684 5029 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 12:51:41.033314 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015735 5029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 12:51:41.033741 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015742 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 12:51:41.033741 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015746 5029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 12:51:41.033741 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015751 5029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 12:51:41.033741 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015755 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 12:51:41.033741 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015759 5029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 12:51:41.033741 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015763 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 12:51:41.033741 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015768 5029 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 12:51:41.033741 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015772 5029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 12:51:41.033741 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015777 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 12:51:41.033741 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015781 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 12:51:41.033741 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015785 5029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 12:51:41.033741 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015790 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 12:51:41.033741 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015795 5029 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 12:51:41.033741 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015799 5029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 12:51:41.033741 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015803 5029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 12:51:41.033741 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015808 5029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 12:51:41.033741 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015813 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 12:51:41.033741 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015817 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 12:51:41.033741 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015823 5029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 12:51:41.033741 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015828 5029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 12:51:41.034238 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015832 5029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 12:51:41.034238 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015836 5029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 12:51:41.034238 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015842 5029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 12:51:41.034238 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015846 5029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 12:51:41.034238 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015851 5029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 12:51:41.034238 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015856 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 12:51:41.034238 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015861 5029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 12:51:41.034238 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015865 5029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 12:51:41.034238 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015869 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 12:51:41.034238 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015873 5029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 12:51:41.034238 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015878 5029 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 12:51:41.034238 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015882 5029 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 12:51:41.034238 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015886 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 12:51:41.034238 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015890 5029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 12:51:41.034238 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015895 5029 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 12:51:41.034238 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015899 5029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 12:51:41.034238 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015904 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 12:51:41.034238 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015908 5029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 12:51:41.034238 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015913 5029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 12:51:41.034238 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015918 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 12:51:41.034753 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015922 5029 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 12:51:41.034753 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015926 5029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 12:51:41.034753 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015930 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 12:51:41.034753 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015935 5029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 12:51:41.034753 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015939 5029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 12:51:41.034753 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015944 5029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 12:51:41.034753 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015949 5029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 12:51:41.034753 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015953 5029 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 12:51:41.034753 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015958 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 12:51:41.034753 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015962 5029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 12:51:41.034753 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015969 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 12:51:41.034753 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015973 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 12:51:41.034753 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015977 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 12:51:41.034753 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015981 5029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 12:51:41.034753 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015988 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 12:51:41.034753 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015992 5029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 12:51:41.034753 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.015997 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 12:51:41.034753 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016001 5029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 12:51:41.034753 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016005 5029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 12:51:41.034753 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016010 5029 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 12:51:41.035242 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016014 5029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 12:51:41.035242 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016020 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 12:51:41.035242 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016023 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 12:51:41.035242 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016028 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 12:51:41.035242 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016032 5029 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 12:51:41.035242 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016036 5029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 12:51:41.035242 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016040 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 12:51:41.035242 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016044 5029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 12:51:41.035242 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016048 5029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 12:51:41.035242 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016052 5029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 12:51:41.035242 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016056 5029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 12:51:41.035242 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016061 5029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 12:51:41.035242 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016081 5029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 12:51:41.035242 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016085 5029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 12:51:41.035242 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016088 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 12:51:41.035242 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016092 5029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 12:51:41.035242 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016096 5029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 12:51:41.035242 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016099 5029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 12:51:41.035242 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016104 5029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 12:51:41.035242 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016109 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 12:51:41.035725 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016112 5029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 12:51:41.035725 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016116 5029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 12:51:41.035725 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016121 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 12:51:41.035725 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016125 5029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 12:51:41.035725 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016129 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 12:51:41.035725 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016132 5029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 12:51:41.035725 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016138 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 12:51:41.035725 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016142 5029 feature_gate.go:386] unrecognized feature gate: Example Oct 30 12:51:41.035725 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016145 5029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 12:51:41.035725 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016150 5029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 12:51:41.035725 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016153 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 12:51:41.035725 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016157 5029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 12:51:41.035725 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016163 5029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 12:51:41.035725 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016168 5029 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 12:51:41.035725 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016172 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 12:51:41.035725 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016177 5029 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 12:51:41.035725 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016181 5029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 12:51:41.035725 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016185 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 12:51:41.035725 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016191 5029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 12:51:41.036195 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016197 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 12:51:41.036195 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016202 5029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 12:51:41.036195 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016207 5029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 12:51:41.036195 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016212 5029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 12:51:41.036195 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016217 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 12:51:41.036195 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:41.016223 5029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 12:51:41.036195 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.016309 5029 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 12:51:41.036195 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.017799 5029 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Oct 30 12:51:41.036195 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.017872 5029 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Oct 30 12:51:41.036195 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.018163 5029 server.go:1014] "Starting client certificate rotation" Oct 30 12:51:41.036195 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.018348 5029 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 12:51:41.036195 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.018399 5029 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 12:46:00 +0000 UTC" deadline="2028-06-09 14:36:02.623187061 +0000 UTC" Oct 30 12:51:41.036195 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.018456 5029 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14113h44m21.604733379s" Oct 30 12:51:41.036195 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.018895 5029 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 12:51:41.036195 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.020696 5029 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 12:51:41.036195 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.022015 5029 log.go:25] "Validated CRI v1 runtime API" Oct 30 12:51:41.036622 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.024110 5029 log.go:25] "Validated CRI v1 image API" Oct 30 12:51:41.036622 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.024821 5029 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 12:51:41.036622 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.026729 5029 fs.go:135] Filesystem UUIDs: map[280c0489-71da-4ac0-a832-7b90884f3ed2:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 95de3539-e8a7-433c-947e-18b76503bfb5:/dev/nvme0n1p3] Oct 30 12:51:41.036732 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.026749 5029 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0aad8a9e73dea6fa13518239e610366e206fdf55a94179fa84e3f08d34e96599/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0aad8a9e73dea6fa13518239e610366e206fdf55a94179fa84e3f08d34e96599/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0c0a7e5307867f709ccc4a87d9633f02f62862e18e356e3011abb8924c16d1db/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0c0a7e5307867f709ccc4a87d9633f02f62862e18e356e3011abb8924c16d1db/userdata/shm major:0 minor:296 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/20283987a635d95c8db1514b2f3ef49bf514699f23b3ad259d918ae38cde43f5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/20283987a635d95c8db1514b2f3ef49bf514699f23b3ad259d918ae38cde43f5/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/24302e1400dcaebd4cb16e81d45385e0c915c341c9816031cf4b7e45c2174779/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/24302e1400dcaebd4cb16e81d45385e0c915c341c9816031cf4b7e45c2174779/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/293b3487250d7ffce1669a266424a8db29e6a9d7bf5d24e4d36e5c0fecd1ef44/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/293b3487250d7ffce1669a266424a8db29e6a9d7bf5d24e4d36e5c0fecd1ef44/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/31b370d6fd236b02a8a8fbd660474aaa1fa31699add3ccee185e54d698bddad0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/31b370d6fd236b02a8a8fbd660474aaa1fa31699add3ccee185e54d698bddad0/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/42e3b567f84d63466cb877bf58cc08c2897a625926e6e52a4230fc9efeafa857/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/42e3b567f84d63466cb877bf58cc08c2897a625926e6e52a4230fc9efeafa857/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5d058b5a3ab026a3bb963d88805fdecb2c35b5a44bd7181f921f4a1ac848f65f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5d058b5a3ab026a3bb963d88805fdecb2c35b5a44bd7181f921f4a1ac848f65f/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5d9c63e84cebf9646cbd5eebebf0de9fd16426bee0278ac370280e367efef2c6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5d9c63e84cebf9646cbd5eebebf0de9fd16426bee0278ac370280e367efef2c6/userdata/shm major:0 minor:229 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/64bdaa665cabf5cc4b070b756ef76602bdfeadbc86d142af3996b25b23e652cd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/64bdaa665cabf5cc4b070b756ef76602bdfeadbc86d142af3996b25b23e652cd/userdata/shm major:0 minor:216 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/666dded619d9762c36a9a0c09fe3451d8392977966f7dc2aeecb6ccac98bbc49/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/666dded619d9762c36a9a0c09fe3451d8392977966f7dc2aeecb6ccac98bbc49/userdata/shm major:0 minor:342 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/683b24075513d15c397f91afaf5097736db18b34fbf3597da3ac3094eac8cc95/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/683b24075513d15c397f91afaf5097736db18b34fbf3597da3ac3094eac8cc95/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/68be897524ba08439268cdfb7db77cfeb418946d1e58c1c8faca16f7e39ac5bb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/68be897524ba08439268cdfb7db77cfeb418946d1e58c1c8faca16f7e39ac5bb/userdata/shm major:0 minor:293 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6c71368cbc5b1b9f6817e4e8f210e1091f6386525d41a39c6d28f280459c8a9a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6c71368cbc5b1b9f6817e4e8f210e1091f6386525d41a39c6d28f280459c8a9a/userdata/shm major:0 minor:340 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6cf91b452eede6076a34dcea2736ad13d357a8050ee4e321a6c9ae82f3890534/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6cf91b452eede6076a34dcea2736ad13d357a8050ee4e321a6c9ae82f3890534/userdata/shm major:0 minor:219 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8a211458f90a22000e7c1f0683391f60671e19098dee129dbfbc3fee70f7a3b0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8a211458f90a22000e7c1f0683391f60671e19098dee129dbfbc3fee70f7a3b0/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9231d3aca1a5eada24e65c5d97f815cd61a12bf38d58e2285a9c0a06d074d3da/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9231d3aca1a5eada24e65c5d97f815cd61a12bf38d58e2285a9c0a06d074d3da/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ae6e257ce29f7a6b8561fca5019d8978ad0f257f6e5e609b98d1c5d3eb418a16/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ae6e257ce29f7a6b8561fca5019d8978ad0f257f6e5e609b98d1c5d3eb418a16/userdata/shm major:0 minor:328 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c0b42f36f921b49fec4166c8f47d231869253e78239ef978d91924ed1d90d1ae/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c0b42f36f921b49fec4166c8f47d231869253e78239ef978d91924ed1d90d1ae/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c305a6af368aa3db922feb23caa6802fb98ab51beca667b418ee32ea7d4c79dd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c305a6af368aa3db922feb23caa6802fb98ab51beca667b418ee32ea7d4c79dd/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c44099040f0f9dbde74f7f75f936ba3c25726e5142e589d606fb6828685e3a2f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c44099040f0f9dbde74f7f75f936ba3c25726e5142e589d606fb6828685e3a2f/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c84db34261e5eaa6a7c9f6a75c2bc81bfc62f530f40cf3ac52f021f126ddd617/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c84db34261e5eaa6a7c9f6a75c2bc81bfc62f530f40cf3ac52f021f126ddd617/userdata/shm major:0 minor:95 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ee279992e3ac50ef6da1a2f63883c9d08d538b348a1eb641a44e81c9aa3da5fc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ee279992e3ac50ef6da1a2f63883c9d08d538b348a1eb641a44e81c9aa3da5fc/userdata/shm major:0 minor:215 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f266024b642e8c0a51cea8631c67704b363aa8bd6ba2b134d581bf3e42ccfc5a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f266024b642e8c0a51cea8631c67704b363aa8bd6ba2b134d581bf3e42ccfc5a/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f7bd715f51c9289bcb3cd2d9c8f31ed55e2817493c4c2ad5e461b98dd43336fe/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f7bd715f51c9289bcb3cd2d9c8f31ed55e2817493c4c2ad5e461b98dd43336fe/userdata/shm major:0 minor:220 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0b1634e9-3028-4b6a-b651-9155018b3c5a/volumes/kubernetes.io~projected/kube-api-access-997qf:{mountpoint:/var/lib/kubelet/pods/0b1634e9-3028-4b6a-b651-9155018b3c5a/volumes/kubernetes.io~projected/kube-api-access-997qf major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0d6a61ca-a15f-4742-8e81-6057c23e51c8/volumes/kubernetes.io~projected/kube-api-access-5djfp:{mountpoint:/var/lib/kubelet/pods/0d6a61ca-a15f-4742-8e81-6057c23e51c8/volumes/kubernetes.io~projected/kube-api-access-5djfp major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0e13113b-b4d2-42d7-9e1c-642a790bcd55/volumes/kubernetes.io~projected/kube-api-access-j8ksd:{mountpoint:/var/lib/kubelet/pods/0e13113b-b4d2-42d7-9e1c-642a790bcd55/volumes/kubernetes.io~projected/kube-api-access-j8ksd major:0 minor:237 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/16a02457-151b-4e96-8118-e7695190b9be/volumes/kubernetes.io~projected/kube-api-access-dc5nm:{mountpoint:/var/lib/kubelet/pods/16a02457-151b-4e96-8118-e7695190b9be/volumes/kubernetes.io~projected/kube-api-access-dc5nm major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1734d386-0cb1-4002-a954-b76b3cf62eaf/volumes/kubernetes.io~projected/kube-api-access-j4v4t:{mountpoint:/var/lib/kubelet/pods/1734d386-0cb1-4002-a954-b76b3cf62eaf/volumes/kubernetes.io~projected/kube-api-access-j4v4t major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a8f7b67-6d17-4f5e-a79b-52481ea0fd48/volumes/kubernetes.io~projected/kube-api-access-srvwc:{mountpoint:/var/lib/kubelet/pods/1a8f7b67-6d17-4f5e-a79b-52481ea0fd48/volumes/kubernetes.io~projected/kube-api-access-srvwc major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/23d95081-1188-4002-997e-65ca57111fab/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/23d95081-1188-4002-997e-65ca57111fab/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/23d95081-1188-4002-997e-65ca57111fab/volumes/kubernetes.io~projected/kube-api-access-j8hrc:{mountpoint:/var/lib/kubelet/pods/23d95081-1188-4002-997e-65ca57111fab/volumes/kubernetes.io~projected/kube-api-access-j8hrc major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/23d95081-1188-4002-997e-65ca57111fab/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/23d95081-1188-4002-997e-65ca57111fab/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2a12192e-f0c8-4354-b9fa-1b65b04ebc19/volumes/kubernetes.io~projected/kube-api-access-mld9s:{mountpoint:/var/lib/kubelet/pods/2a12192e-f0c8-4354-b9fa-1b65b04ebc19/volumes/kubernetes.io~projected/kube-api-access-mld9s major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2f5f56fc-d2d4-4cef-86c0-0ac4b7a51e41/volumes/kubernetes.io~projected/kube-api-access-75kpm:{mountpoint:/var/lib/kubelet/pods/2f5f56fc-d2d4-4cef-86c0-0ac4b7a51e41/volumes/kubernetes.io~projected/kube-api-access-75kpm major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3162c3d2-e86f-4ae1-8196-0414509fae9e/volumes/kubernetes.io~projected/kube-api-access-7vkl7:{mountpoint:/var/lib/kubelet/pods/3162c3d2-e86f-4ae1-8196-0414509fae9e/volumes/kubernetes.io~projected/kube-api-access-7vkl7 major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/38b6bfaf-2fb2-455b-9e7d-32586b3f216f/volumes/kubernetes.io~projected/kube-api-access-sbwv9:{mountpoint:/var/lib/kubelet/pods/38b6bfaf-2fb2-455b-9e7d-32586b3f216f/volumes/kubernetes.io~projected/kube-api-access-sbwv9 major:0 minor:236 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3e77ccc1-14a3-4f85-a214-854bd392733a/volumes/kubernetes.io~projected/kube-api-access-pqfvq:{mountpoint:/var/lib/kubelet/pods/3e77ccc1-14a3-4f85-a214-854bd392733a/volumes/kubernetes.io~projected/kube-api-access-pqfvq major:0 minor:233 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3e77ccc1-14a3-4f85-a214-854bd392733a/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/3e77ccc1-14a3-4f85-a214-854bd392733a/volumes/kubernetes.io~secret/default-certificate major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3e77ccc1-14a3-4f85-a214-854bd392733a/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/3e77ccc1-14a3-4f85-a214-854bd392733a/volumes/kubernetes.io~secret/stats-auth major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/49b68250-16ee-42f1-9730-e4d326c96cd8/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/49b68250-16ee-42f1-9730-e4d326c96cd8/volumes/kubernetes.io~secret/agent-certs major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5133b471-c5f8-4b35-b06c-1d8f429c3ba2/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/5133b471-c5f8-4b35-b06c-1d8f429c3ba2/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5133b471-c5f8-4b35-b06c-1d8f429c3ba2/volumes/kubernetes.io~projected/kube-api-access-gjl5n:{mountpoint:/var/lib/kubelet/pods/5133b471-c5f8-4b35-b06c-1d8f429c3ba2/volumes/kubernetes.io~projected/kube-api-access-gjl5n major:0 minor:235 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5133b471-c5f8-4b35-b06c-1d8f429c3ba2/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/5133b471-c5f8-4b35-b06c-1d8f429c3ba2/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5133b471-c5f8-4b35-b06c-1d8f429c3ba2/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/5133b471-c5f8-4b35-b06c-1d8f429c3ba2/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:228 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3/volumes/kubernetes.io~projected/kube-api-access-d5fjr:{mountpoint:/var/lib/kubelet/pods/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3/volumes/kubernetes.io~projected/kube-api-access-d5fjr major:0 minor:327 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/52ca5902-69f0-462f-ae35-cbbc7f378345/volumes/kubernetes.io~projected/kube-api-access-n7r5b:{mountpoint:/var/lib/kubelet/pods/52ca5902-69f0-462f-ae35-cbbc7f378345/volumes/kubernetes.io~projected/kube-api-access-n7r5b major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/52ca5902-69f0-462f-ae35-cbbc7f378345/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/52ca5902-69f0-462f-ae35-cbbc7f378345/volumes/kubernetes.io~secret/serving-cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5cd3297b-e3fe-40e0-be87-88a67236d9f0/volumes/kubernetes.io~projected/kube-api-access-fhns5:{mountpoint:/var/lib/kubelet/pods/5cd3297b-e3fe-40e0-be87-88a67236d9f0/volumes/kubernetes.io~projected/kube-api-access-fhns5 major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/66992144-51dd-4c1d-95c8-63862bca97cc/volumes/kubernetes.io~projected/kube-api-access-mdmq4:{mountpoint:/var/lib/kubelet/pods/66992144-51dd-4c1d-95c8-63862bca97cc/volumes/kubernetes.io~projected/kube-api-access-mdmq4 major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6d7c203f-f696-471f-baf8-32c0ad647baf/volumes/kubernetes.io~projected/kube-api-access-w928x:{mountpoint:/var/lib/kubelet/pods/6d7c203f-f696-471f-baf8-32c0ad647baf/volumes/kubernetes.io~projected/kube-api-access-w928x major:0 minor:334 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/75bd9ac5-49ff-4c58-85c7-79b181150555/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/75bd9ac5-49ff-4c58-85c7-79b181150555/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:276 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7d93802e-f47d-4319-b308-04fe84901551/volumes/kubernetes.io~projected/kube-api-access-64858:{mountpoint:/var/lib/kubelet/pods/7d93802e-f47d-4319-b308-04fe84901551/volumes/kubernetes.io~projected/kube-api-access-64858 major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d/volumes/kubernetes.io~projected/kube-api-access-mvwvc:{mountpoint:/var/lib/kubelet/pods/7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d/volumes/kubernetes.io~projected/kube-api-access-mvwvc major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/85617466-6b8e-45d4-a45e-e9650078c0be/volumes/kubernetes.io~projected/kube-api-access-9cgzb:{mountpoint:/var/lib/kubelet/pods/85617466-6b8e-45d4-a45e-e9650078c0be/volumes/kubernetes.io~projected/kube-api-access-9cgzb major:0 minor:339 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462/volumes/kubernetes.io~projected/kube-api-access-xg5w5:{mountpoint:/var/lib/kubelet/pods/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462/volumes/kubernetes.io~projected/kube-api-access-xg5w5 major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462/volumes/kubernetes.io~secret/serving-cert major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9951ec02-ee4f-488c-8cab-3985569328f8/volumes/kubernetes.io~projected/kube-api-access-htqvt:{mountpoint:/var/lib/kubelet/pods/9951ec02-ee4f-488c-8cab-3985569328f8/volumes/kubernetes.io~projected/kube-api-access-htqvt major:0 minor:234 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9951ec02-ee4f-488c-8cab-3985569328f8/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/9951ec02-ee4f-488c-8cab-3985569328f8/volumes/kubernetes.io~secret/serving-cert major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43/volumes/kubernetes.io~projected/kube-api-access-q6zpc:{mountpoint:/var/lib/kubelet/pods/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43/volumes/kubernetes.io~projected/kube-api-access-q6zpc major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c4c9c543-f712-4a48-949f-d3f3bc8f77e1/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/c4c9c543-f712-4a48-949f-d3f3bc8f77e1/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c4c9c543-f712-4a48-949f-d3f3bc8f77e1/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/c4c9c543-f712-4a48-949f-d3f3bc8f77e1/volumes/kubernetes.io~empty-dir/tmp major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c4c9c543-f712-4a48-949f-d3f3bc8f77e1/volumes/kubernetes.io~projected/kube-api-access-pmqmx:{mountpoint:/var/lib/kubelet/pods/c4c9c543-f712-4a48-949f-d3f3bc8f77e1/volumes/kubernetes.io~projected/kube-api-access-pmqmx major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d4ca5142-5b8a-4406-9ed0-06f9edea89bc/volumes/kubernetes.io~projected/kube-api-access-5h7bv:{mountpoint:/var/lib/kubelet/pods/d4ca5142-5b8a-4406-9ed0-06f9edea89bc/volumes/kubernetes.io~projected/kube-api-access-5h7bv major:0 minor:232 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d4ca5142-5b8a-4406-9ed0-06f9edea89bc/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/d4ca5142-5b8a-4406-9ed0-06f9edea89bc/volumes/kubernetes.io~secret/cert major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e9446374-a7ea-4f95-baa4-d6dad6805b65/volumes/kubernetes.io~projected/kube-api-access-wd65z:{mountpoint:/var/lib/kubelet/pods/e9446374-a7ea-4f95-baa4-d6dad6805b65/volumes/kubernetes.io~projected/kube-api-access-wd65z major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f0d51a79-823b-4059-94c3-0535fc882a44/volumes/kubernetes.io~projected/kube-api-access-scm4l:{mountpoint:/var/lib/kubelet/pods/f0d51a79-823b-4059-94c3-0535fc882a44/volumes/kubernetes.io~projected/kube-api-access-scm4l major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb047779-ad10-4f7e-a45a-4a154ab60ad8/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/fb047779-ad10-4f7e-a45a-4a154ab60ad8/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb047779-ad10-4f7e-a45a-4a154ab60ad8/volumes/kubernetes.io~projected/kube-api-access-rk88t:{mountpoint:/var/lib/kubelet/pods/fb047779-ad10-4f7e-a45a-4a154ab60ad8/volumes/kubernetes.io~projected/kube-api-access-rk88t major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb047779-ad10-4f7e-a45a-4a154ab60ad8/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/fb047779-ad10-4f7e-a45a-4a154ab60ad8/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb047779-ad10-4f7e-a45a-4a154ab60ad8/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/fb047779-ad10-4f7e-a45a-4a154ab60ad8/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fdddab43-acaf-4153-bb52-3e60044a35dc/volumes/kubernetes.io~projected/kube-api-access-c59v8:{mountpoint:/var/lib/kubelet/pods/fdddab43-acaf-4153-bb52-3e60044a35dc/volumes/kubernetes.io~projected/kube-api-access-c59v8 major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fdddab43-acaf-4153-bb52-3e60044a35dc/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/fdddab43-acaf-4153-bb52-3e60044a35dc/volumes/kubernetes.io~secret/serving-cert major:0 minor:201 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/65807235785d42c2811e5b254f2294179640680bb52765db7c584e89bfc99c97/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/11461906befad8fe865277cfdd60815da7ea63e4f5deeaace6645908e7eca46c/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/7574fa691bf2b9f94ddf5c528c4a5d28d4ab44157321f2c5dedaa38dba983c2d/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/f40a314479dca2fddbfbf3411127bb6f8b80eae227c3a8063736ea80a002f224/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/3fadc8f69106cc0d25fc875bc769de8786bfb26f4313d944eb12946554b8bc9e/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/5302e8ef450f4d7828d512bbd31f2e8d792d7c2bf8f76ca0313b5a431729fb3c/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/cb7152f88a55fd7f8ea7ed98a8743914dfdd34e96be378a38edc41ab285b168d/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/a2d3b9085cc488f0c12153c7283f50e0ef4386f5feba499be681ebe6946991d0/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-133:{mountpoint:/var/lib/containers/storage/overlay/2993253aec4a7dc6120acd9f8ad458a3b1faf07197cdf9c24b2c2a6fe15464a3/merged major:0 minor:133 fsType:overlay blockSize:0} overlay_0-135:{mountpoint:/var/lib/containers/storage/overlay/c4fea42ee56265651a20bb592195aaf19cdf3fa25f0d89e566ed4da3005e5866/merged major:0 minor:135 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/61ddff78ef2462837328d977d86c35d18da5eee5359f13e34f6a088e16ed7441/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/fb54f7f0a9ac526591e254b1f1431cb4224c0868c800dfed0589d16fb3baebb8/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/106c5b082f358518b9ec753d95a06493ff27e69c1cec98c7b2b9fdaac006552c/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/a906d6e6afaa922aced9089e7cae4eb2a7fc02047e070bc070c01a43d4d13aa2/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/f075fcfa435aa61f03e2e3a809bb39b4a3207e3365d125e50bc0b9c63815e437/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/2d95667b0ef94b43764e2ff4a10cb9378f75b221794ffec03d9461e1891f6753/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/adff0fabe97478294bdf9c3540a10217dd7f2a6583acbcd477cebe060c683a4f/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/ac76b105fab6ea41d4bb7dd2373ea6f44b7ec6f704ec5ea961ac395aa05ea61e/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/68f8e474b51bbd8c9ce0b24d4cbe8a3dfab00d32ad78091eece70b8c4f81c314/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/2b7d99c5738b7dbfce97566c83c1aaefdb4089685f52beb096f5139039cafd90/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/dd8659cc353fc1cc86fb3f2ea75ee9f187868edf29f7e3d033ff915c65102423/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/644bec856f2cf03f967f3254902e64e558c27fb08a393461abc011b35f4be490/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/32c749e08a1e00a815223463be67ab8f856a7a7c5ac233ca770f7b76e90a9bc7/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/b084609845df188ab9e596bc2b1ffb345b7b7949441d2c12593f739b9921c2ae/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/8bf33f515d08c9dd705d08be348655b5270691ffbf95f43ca43448446e174d24/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/29d3f5be0dcb6cd9d80ffd8ce6cb6c5b9deed4d4ec6313e6f3da4f8718622f02/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/4e15c0be32f874a2137d6cf33a59a7c79c4eba544a2fe64502273b049f439f0e/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/5053131e880e5594f4308a3898a9021276eac8757bd3584b55b71687e346b79a/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/2a63e6b9fb79f15564f5a7c6bde1be2e339b882db7dc41f35be47904b343cc6c/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/090f3f5daf611e635dec7719739c76514c814806ea3295e96426967d644ba8c3/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/c4ea313805888e3c516f418518694b1c907b97a2576d96ff269cde277876b448/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/212f93b7fea4a1b03150d842667375a49db0cb2ede874c4d5a758e6cdb81886e/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-270:{mountpoint:/var/lib/containers/storage/overlay/9c2a76f945ad7d7dca173333931679bd433ee3a66c31049df2ffc077548fc640/merged major:0 minor:270 fsType:overlay blockSize:0} overlay_0-272:{mountpoint:/var/lib/containers/storage/overlay/42867e6d5adb052e30a4f898dc06ce82f3c12d6746b45c74c8ca0d347be78fbf/merged major:0 minor:272 fsType:overlay blockSize:0} overlay_0-274:{mountpoint:/var/lib/containers/storage/overlay/837be3d959721e2402a96a99e44be5fb615430cb5b0fb7005619ef97ac6a677c/merged major:0 minor:274 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/60e9c051f758a0f71ca884f3787b5a62fd0447dab9b2c2910423d4467abf6f37/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/ded04706f0787405180874aa961257236745af582895a3ebbaa2a87d41effe4b/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/95be9e560b45847577c7c036e58bef3b98cd698c9fd3946e1cf7d68456a9d96c/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/c270f462f68e393cf0fa4bd30b049d48ab24a323aaeb69b411ad3e4cbe8a405e/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-306:{mountpoint:/var/lib/containers/storage/overlay/d586a87426be9eef55922575fb11778f6a335521b2dc680fba3dd2153a98febc/merged major:0 minor:306 fsType:overlay blockSize:0} overlay_0-312:{mountpoint:/var/lib/containers/storage/overlay/ecda527760de5c3247ee0f37a5f4a664fec5fc5da6f95a90a4c807924a3e7f78/merged major:0 minor:312 fsType:overlay blockSize:0} overlay_0-330:{mountpoint:/var/lib/containers/storage/overlay/70f2c8b3524fd77485d1a3ee9cd72062d2ceb82e4c3b202039c10f09492852f3/merged major:0 minor:330 fsType:overlay blockSize:0} overlay_0-332:{mountpoint:/var/lib/containers/storage/overlay/e900c7f86d4a52e658f3869e6b27b03074c55294abd63222dbeee38f2d010222/merged major:0 minor:332 fsType:overlay blockSize:0} overlay_0-344:{mountpoint:/var/lib/containers/storage/overlay/aa9c2bb327b3504ef7700782f66a2e580bef64b39aa968a2f41f5b5babc8705f/merged major:0 minor:344 fsType:overlay blockSize:0} overlay_0-346:{mountpoint:/var/lib/containers/storage/overlay/77805874d45b8d116c2f41182cc6086bbc530703ab584547aa53e9b948a8a143/merged major:0 minor:346 fsType:overlay blockSize:0} overlay_0-352:{mountpoint:/var/lib/containers/storage/overlay/1fcb45f3d9d0a25bdbc043cf0401455153bad3fbadb53d88d729a6031b43cbb6/merged major:0 minor:352 fsType:overlay blockSize:0} overlay_0-354:{mountpoint:/var/lib/containers/storage/overlay/b61744790bc177b1968cc9a9b1cec06855cbe46164246e8ad12f57f7d5b3e358/merged major:0 minor:354 fsType:overlay blockSize:0} overlay_0-356:{mountpoint:/var/lib/containers/storage/overlay/8321768f33968cda8e7459bf746f63ed6ff7b397b7ba45e389336284d0d49895/merged major:0 minor:356 fsType:overlay blockSize:0} overlay_0-362:{mountpoint:/var/lib/containers/storage/overlay/a1843960e6a1c01b592e216340e1592202373ebc55c4797f21ce0b81d343544b/merged major:0 minor:362 fsType:overlay blockSize:0} overlay_0-364:{mountpoint:/var/lib/containers/storage/overlay/4685c0956a8592f9163c929159eb00ab40c5bd0bbe78374fea926b38c30008d9/merged major:0 minor:364 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/e04ef8d29dd2e19d0dd49fd0044b8a0e825dfaaa88cf3f8535690cf6eedd77f5/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/d438e7a9cfb3157c85c5a27ec2f1797ce489466493d013de232e7e86fbc1590f/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/ab5dbaf3b0d7543def240a31e12b26750fea119b5a3421cf8ef18846f4d9cbd0/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/b1a35c916fe382f594114d6ae4a31df542ce0e999a286b4c86c6bce05148bbf4/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/4ef3ed9ac85b43b84cc050875d6c01c0999432aa67d91c70341f88821db472e2/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/0725baf99337351888ce78371faa19c4d88b6330e54b5ec130c558bacd6e3589/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/f3a87c5b704b8b178e4e5f2290c241a003637b6657281cf46924434ceb91915a/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/b9f4607ef204b166029d85018c3c116e10f6f610ce456b31fd91ec2906034442/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/5d1e9ef10d6aa0f24c1cd363143b92451d6722511e51cd5a9e5ab6e87a032675/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-98:{mountpoint:/var/lib/containers/storage/overlay/91ac2a5c6222de0cce38088537f6247ae34c6556395ff8585a6ffb998ba20ea3/merged major:0 minor:98 fsType:overlay blockSize:0}] Oct 30 12:51:41.036953 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.035990 5029 manager.go:217] Machine: {Timestamp:2026-10-30 12:51:41.035533825 +0000 UTC m=+0.061175064 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099942 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29b418043af34e36e028f2a0b27fb6 SystemUUID:ec29b418-043a-f34e-36e0-28f2a0b27fb6 BootID:b3e9cffb-e49e-42f3-845e-a30a8ecc9933 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/66992144-51dd-4c1d-95c8-63862bca97cc/volumes/kubernetes.io~projected/kube-api-access-mdmq4 DeviceMajor:0 DeviceMinor:207 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f7bd715f51c9289bcb3cd2d9c8f31ed55e2817493c4c2ad5e461b98dd43336fe/userdata/shm DeviceMajor:0 DeviceMinor:220 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-344 DeviceMajor:0 DeviceMinor:344 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-133 DeviceMajor:0 DeviceMinor:133 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fb047779-ad10-4f7e-a45a-4a154ab60ad8/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-354 DeviceMajor:0 DeviceMinor:354 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/64bdaa665cabf5cc4b070b756ef76602bdfeadbc86d142af3996b25b23e652cd/userdata/shm DeviceMajor:0 DeviceMinor:216 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c305a6af368aa3db922feb23caa6802fb98ab51beca667b418ee32ea7d4c79dd/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-356 DeviceMajor:0 DeviceMinor:356 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/293b3487250d7ffce1669a266424a8db29e6a9d7bf5d24e4d36e5c0fecd1ef44/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c4c9c543-f712-4a48-949f-d3f3bc8f77e1/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c44099040f0f9dbde74f7f75f936ba3c25726e5142e589d606fb6828685e3a2f/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fdddab43-acaf-4153-bb52-3e60044a35dc/volumes/kubernetes.io~projected/kube-api-access-c59v8 DeviceMajor:0 DeviceMinor:211 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3e77ccc1-14a3-4f85-a214-854bd392733a/volumes/kubernetes.io~projected/kube-api-access-pqfvq DeviceMajor:0 DeviceMinor:233 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-272 DeviceMajor:0 DeviceMinor:272 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/23d95081-1188-4002-997e-65ca57111fab/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6cf91b452eede6076a34dcea2736ad13d357a8050ee4e321a6c9ae82f3890534/userdata/shm DeviceMajor:0 DeviceMinor:219 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-364 DeviceMajor:0 DeviceMinor:364 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e9446374-a7ea-4f95-baa4-d6dad6805b65/volumes/kubernetes.io~projected/kube-api-access-wd65z DeviceMajor:0 DeviceMinor:64 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fb047779-ad10-4f7e-a45a-4a154ab60ad8/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:212 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/5133b471-c5f8-4b35-b06c-1d8f429c3ba2/volumes/kubernetes.io~projected/kube-api-access-gjl5n DeviceMajor:0 DeviceMinor:235 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/20283987a635d95c8db1514b2f3ef49bf514699f23b3ad259d918ae38cde43f5/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d/volumes/kubernetes.io~projected/kube-api-access-mvwvc DeviceMajor:0 DeviceMinor:60 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0b1634e9-3028-4b6a-b651-9155018b3c5a/volumes/kubernetes.io~projected/kube-api-access-997qf DeviceMajor:0 DeviceMinor:62 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2a12192e-f0c8-4354-b9fa-1b65b04ebc19/volumes/kubernetes.io~projected/kube-api-access-mld9s DeviceMajor:0 DeviceMinor:63 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fb047779-ad10-4f7e-a45a-4a154ab60ad8/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:203 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d4ca5142-5b8a-4406-9ed0-06f9edea89bc/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:222 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3e77ccc1-14a3-4f85-a214-854bd392733a/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:223 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9951ec02-ee4f-488c-8cab-3985569328f8/volumes/kubernetes.io~projected/kube-api-access-htqvt DeviceMajor:0 DeviceMinor:234 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/49b68250-16ee-42f1-9730-e4d326c96cd8/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/42e3b567f84d63466cb877bf58cc08c2897a625926e6e52a4230fc9efeafa857/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9951ec02-ee4f-488c-8cab-3985569328f8/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:226 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8a211458f90a22000e7c1f0683391f60671e19098dee129dbfbc3fee70f7a3b0/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/23d95081-1188-4002-997e-65ca57111fab/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/52ca5902-69f0-462f-ae35-cbbc7f378345/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:202 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43/volumes/kubernetes.io~projected/kube-api-access-q6zpc DeviceMajor:0 DeviceMinor:295 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c0b42f36f921b49fec4166c8f47d231869253e78239ef978d91924ed1d90d1ae/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/31b370d6fd236b02a8a8fbd660474aaa1fa31699add3ccee185e54d698bddad0/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3162c3d2-e86f-4ae1-8196-0414509fae9e/volumes/kubernetes.io~projected/kube-api-access-7vkl7 DeviceMajor:0 DeviceMinor:65 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/16a02457-151b-4e96-8118-e7695190b9be/volumes/kubernetes.io~projected/kube-api-access-dc5nm DeviceMajor:0 DeviceMinor:67 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5133b471-c5f8-4b35-b06c-1d8f429c3ba2/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:221 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462/volumes/kubernetes.io~projected/kube-api-access-xg5w5 DeviceMajor:0 DeviceMinor:205 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fb047779-ad10-4f7e-a45a-4a154ab60ad8/volumes/kubernetes.io~projected/kube-api-access-rk88t DeviceMajor:0 DeviceMinor:213 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5d058b5a3ab026a3bb963d88805fdecb2c35b5a44bd7181f921f4a1ac848f65f/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9231d3aca1a5eada24e65c5d97f815cd61a12bf38d58e2285a9c0a06d074d3da/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/68be897524ba08439268cdfb7db77cfeb418946d1e58c1c8faca16f7e39ac5bb/userdata/shm DeviceMajor:0 DeviceMinor:293 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0d6a61ca-a15f-4742-8e81-6057c23e51c8/volumes/kubernetes.io~projected/kube-api-access-5djfp DeviceMajor:0 DeviceMinor:61 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/24302e1400dcaebd4cb16e81d45385e0c915c341c9816031cf4b7e45c2174779/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1a8f7b67-6d17-4f5e-a79b-52481ea0fd48/volumes/kubernetes.io~projected/kube-api-access-srvwc DeviceMajor:0 DeviceMinor:208 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/23d95081-1188-4002-997e-65ca57111fab/volumes/kubernetes.io~projected/kube-api-access-j8hrc DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/683b24075513d15c397f91afaf5097736db18b34fbf3597da3ac3094eac8cc95/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-274 DeviceMajor:0 DeviceMinor:274 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/75bd9ac5-49ff-4c58-85c7-79b181150555/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:276 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-306 DeviceMajor:0 DeviceMinor:306 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-270 DeviceMajor:0 DeviceMinor:270 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-312 DeviceMajor:0 DeviceMinor:312 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-330 DeviceMajor:0 DeviceMinor:330 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c4c9c543-f712-4a48-949f-d3f3bc8f77e1/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5cd3297b-e3fe-40e0-be87-88a67236d9f0/volumes/kubernetes.io~projected/kube-api-access-fhns5 DeviceMajor:0 DeviceMinor:322 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ae6e257ce29f7a6b8561fca5019d8978ad0f257f6e5e609b98d1c5d3eb418a16/userdata/shm DeviceMajor:0 DeviceMinor:328 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/c4c9c543-f712-4a48-949f-d3f3bc8f77e1/volumes/kubernetes.io~projected/kube-api-access-pmqmx DeviceMajor:0 DeviceMinor:66 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/52ca5902-69f0-462f-ae35-cbbc7f378345/volumes/kubernetes.io~projected/kube-api-access-n7r5b DeviceMajor:0 DeviceMinor:209 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3e77ccc1-14a3-4f85-a214-854bd392733a/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:225 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/5133b471-c5f8-4b35-b06c-1d8f429c3ba2/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:230 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6d7c203f-f696-471f-baf8-32c0ad647baf/volumes/kubernetes.io~projected/kube-api-access-w928x DeviceMajor:0 DeviceMinor:334 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0c0a7e5307867f709ccc4a87d9633f02f62862e18e356e3011abb8924c16d1db/userdata/shm DeviceMajor:0 DeviceMinor:296 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-98 DeviceMajor:0 DeviceMinor:98 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5d9c63e84cebf9646cbd5eebebf0de9fd16426bee0278ac370280e367efef2c6/userdata/shm DeviceMajor:0 DeviceMinor:229 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3/volumes/kubernetes.io~projected/kube-api-access-d5fjr DeviceMajor:0 DeviceMinor:327 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f266024b642e8c0a51cea8631c67704b363aa8bd6ba2b134d581bf3e42ccfc5a/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ee279992e3ac50ef6da1a2f63883c9d08d538b348a1eb641a44e81c9aa3da5fc/userdata/shm DeviceMajor:0 DeviceMinor:215 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/5133b471-c5f8-4b35-b06c-1d8f429c3ba2/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:228 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-135 DeviceMajor:0 DeviceMinor:135 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fdddab43-acaf-4153-bb52-3e60044a35dc/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2f5f56fc-d2d4-4cef-86c0-0ac4b7a51e41/volumes/kubernetes.io~projected/kube-api-access-75kpm DeviceMajor:0 DeviceMinor:204 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/f0d51a79-823b-4059-94c3-0535fc882a44/volumes/kubernetes.io~projected/kube-api-access-scm4l DeviceMajor:0 DeviceMinor:214 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/d4ca5142-5b8a-4406-9ed0-06f9edea89bc/volumes/kubernetes.io~projected/kube-api-access-5h7bv DeviceMajor:0 DeviceMinor:232 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/85617466-6b8e-45d4-a45e-e9650078c0be/volumes/kubernetes.io~projected/kube-api-access-9cgzb DeviceMajor:0 DeviceMinor:339 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-346 DeviceMajor:0 DeviceMinor:346 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-352 DeviceMajor:0 DeviceMinor:352 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-362 DeviceMajor:0 DeviceMinor:362 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c84db34261e5eaa6a7c9f6a75c2bc81bfc62f530f40cf3ac52f021f126ddd617/userdata/shm DeviceMajor:0 DeviceMinor:95 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-332 DeviceMajor:0 DeviceMinor:332 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/666dded619d9762c36a9a0c09fe3451d8392977966f7dc2aeecb6ccac98bbc49/userdata/shm DeviceMajor:0 DeviceMinor:342 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7d93802e-f47d-4319-b308-04fe84901551/volumes/kubernetes.io~projected/kube-api-access-64858 DeviceMajor:0 DeviceMinor:206 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1734d386-0cb1-4002-a954-b76b3cf62eaf/volumes/kubernetes.io~projected/kube-api-access-j4v4t DeviceMajor:0 DeviceMinor:210 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/0e13113b-b4d2-42d7-9e1c-642a790bcd55/volumes/kubernetes.io~projected/kube-api-access-j8ksd DeviceMajor:0 DeviceMinor:237 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6c71368cbc5b1b9f6817e4e8f210e1091f6386525d41a39c6d28f280459c8a9a/userdata/shm DeviceMajor:0 DeviceMinor:340 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/38b6bfaf-2fb2-455b-9e7d-32586b3f216f/volumes/kubernetes.io~projected/kube-api-access-sbwv9 DeviceMajor:0 DeviceMinor:236 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0aad8a9e73dea6fa13518239e610366e206fdf55a94179fa84e3f08d34e96599/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:0aad8a9e73dea6f MacAddress:3e:a1:eb:9c:30:14 Speed:10000 Mtu:8901} {Name:0c0a7e5307867f7 MacAddress:3e:42:2a:6c:dd:98 Speed:10000 Mtu:8901} {Name:20283987a635d95 MacAddress:46:8a:5b:0b:e9:34 Speed:10000 Mtu:8901} {Name:5d9c63e84cebf96 MacAddress:36:01:d3:46:7e:29 Speed:10000 Mtu:8901} {Name:64bdaa665cabf5c MacAddress:b2:c1:ca:02:b1:5f Speed:10000 Mtu:8901} {Name:666dded619d9762 MacAddress:66:4b:79:89:cd:28 Speed:10000 Mtu:8901} {Name:68be897524ba084 MacAddress:6e:23:37:e9:3d:96 Speed:10000 Mtu:8901} {Name:6c71368cbc5b1b9 MacAddress:82:bc:d7:7b:c4:5e Speed:10000 Mtu:8901} {Name:6cf91b452eede60 MacAddress:e6:b9:f2:ce:1c:b5 Speed:10000 Mtu:8901} {Name:8a211458f90a220 MacAddress:46:ef:54:e5:6f:f0 Speed:10000 Mtu:8901} {Name:9231d3aca1a5ead MacAddress:12:ea:f3:19:e1:9e Speed:10000 Mtu:8901} {Name:ae6e257ce29f7a6 MacAddress:46:88:be:6a:0e:a5 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:f2:26:71:c9 Speed:0 Mtu:9001} {Name:br-int MacAddress:7a:ee:1c:62:f3:e3 Speed:0 Mtu:8901} {Name:c305a6af368aa3d MacAddress:c6:6a:9c:0e:95:e3 Speed:10000 Mtu:8901} {Name:ee279992e3ac50e MacAddress:1e:95:5d:e6:a1:34 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:f2:26:71:c9 Speed:0 Mtu:9001} {Name:f7bd715f51c9289 MacAddress:ce:12:be:dc:7b:0d Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:12:35:ba:71:34:d8 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:ce:46:7b:a5:14:90 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 12:51:41.036953 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.036369 5029 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 12:51:41.036953 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.036420 5029 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 12:51:41.036953 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.036668 5029 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 12:51:41.036953 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.036688 5029 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-147.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 12:51:41.036953 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.036817 5029 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 12:51:41.036953 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.036825 5029 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 12:51:41.036953 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.036839 5029 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 12:51:41.036953 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.036852 5029 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 12:51:41.036953 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.036871 5029 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 12:51:41.037571 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.037010 5029 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 12:51:41.037571 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.037136 5029 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 12:51:41.037571 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.037170 5029 kubelet.go:493] "Attempting to sync node with API server" Oct 30 12:51:41.037571 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.037179 5029 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 12:51:41.037571 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.037190 5029 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 12:51:41.037571 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.037198 5029 kubelet.go:405] "Adding apiserver pod source" Oct 30 12:51:41.037571 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.037207 5029 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 12:51:41.037929 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.037913 5029 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 12:51:41.037929 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.037927 5029 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 12:51:41.038430 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.038406 5029 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 12:51:41.040835 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.040818 5029 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 12:51:41.040835 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.040832 5029 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 12:51:41.040954 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.040923 5029 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Oct 30 12:51:41.041123 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.041111 5029 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 12:51:41.041162 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.041140 5029 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 12:51:41.041242 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.041232 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 12:51:41.041281 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.041247 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 12:51:41.041281 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.041253 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 12:51:41.041281 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.041259 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 12:51:41.041281 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.041264 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 12:51:41.041281 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.041270 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 12:51:41.041281 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.041276 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 12:51:41.041281 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.041281 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 12:51:41.041609 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.041287 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 12:51:41.041609 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.041293 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 12:51:41.041609 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.041301 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 12:51:41.041609 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.041310 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 12:51:41.041609 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.041335 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 12:51:41.041609 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.041340 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 12:51:41.044777 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.044762 5029 server.go:1263] "Started kubelet" Oct 30 12:51:41.045461 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.044859 5029 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 12:51:41.045461 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.044846 5029 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 12:51:41.045461 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.044922 5029 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 12:51:41.045368 ip-10-0-141-147 systemd[1]: Started Kubernetes Kubelet. Oct 30 12:51:41.045777 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.045759 5029 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 12:51:41.048971 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.046756 5029 server.go:317] "Adding debug handlers to kubelet server" Oct 30 12:51:41.049719 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.049702 5029 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 12:51:41.049845 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.049749 5029 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 12:51:41.049845 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.049803 5029 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 12:46:02 +0000 UTC" deadline="2028-06-01 09:02:53.233442572 +0000 UTC" Oct 30 12:51:41.049845 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.049828 5029 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 12:51:41.049845 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.049827 5029 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13916h11m12.183618146s" Oct 30 12:51:41.049845 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.049814 5029 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 12:51:41.050190 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.049856 5029 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 12:51:41.050322 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:41.050288 5029 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-147.ec2.internal\" not found" Oct 30 12:51:41.051689 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.051340 5029 factory.go:55] Registering systemd factory Oct 30 12:51:41.051689 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.051360 5029 factory.go:223] Registration of the systemd container factory successfully Oct 30 12:51:41.051821 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.051730 5029 factory.go:153] Registering CRI-O factory Oct 30 12:51:41.051821 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.051745 5029 factory.go:223] Registration of the crio container factory successfully Oct 30 12:51:41.051821 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.051797 5029 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 12:51:41.051975 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.051822 5029 factory.go:103] Registering Raw factory Oct 30 12:51:41.051975 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.051843 5029 manager.go:1199] Started watching for new ooms in manager Oct 30 12:51:41.052207 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.052191 5029 manager.go:319] Starting recovery of all containers Oct 30 12:51:41.053911 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.053870 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2f5f56fc-d2d4-4cef-86c0-0ac4b7a51e41" volumeName="kubernetes.io/projected/2f5f56fc-d2d4-4cef-86c0-0ac4b7a51e41-kube-api-access-75kpm" seLinuxMountContext="" Oct 30 12:51:41.054005 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.053911 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="38b6bfaf-2fb2-455b-9e7d-32586b3f216f" volumeName="kubernetes.io/projected/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-kube-api-access-sbwv9" seLinuxMountContext="" Oct 30 12:51:41.054005 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.053926 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3e77ccc1-14a3-4f85-a214-854bd392733a" volumeName="kubernetes.io/projected/3e77ccc1-14a3-4f85-a214-854bd392733a-kube-api-access-pqfvq" seLinuxMountContext="" Oct 30 12:51:41.054005 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.053941 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d7c203f-f696-471f-baf8-32c0ad647baf" volumeName="kubernetes.io/projected/6d7c203f-f696-471f-baf8-32c0ad647baf-kube-api-access-w928x" seLinuxMountContext="" Oct 30 12:51:41.054005 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.053961 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462" volumeName="kubernetes.io/configmap/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-trusted-ca-bundle" seLinuxMountContext="" Oct 30 12:51:41.054005 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.053973 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2a12192e-f0c8-4354-b9fa-1b65b04ebc19" volumeName="kubernetes.io/configmap/2a12192e-f0c8-4354-b9fa-1b65b04ebc19-serviceca" seLinuxMountContext="" Oct 30 12:51:41.054005 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.053985 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462" volumeName="kubernetes.io/secret/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-serving-cert" seLinuxMountContext="" Oct 30 12:51:41.054005 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.053997 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23d95081-1188-4002-997e-65ca57111fab" volumeName="kubernetes.io/projected/23d95081-1188-4002-997e-65ca57111fab-kube-api-access-j8hrc" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054011 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5133b471-c5f8-4b35-b06c-1d8f429c3ba2" volumeName="kubernetes.io/secret/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-installation-pull-secrets" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054025 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462" volumeName="kubernetes.io/empty-dir/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-tmp" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054037 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4c9c543-f712-4a48-949f-d3f3bc8f77e1" volumeName="kubernetes.io/empty-dir/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-tuned" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054049 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb047779-ad10-4f7e-a45a-4a154ab60ad8" volumeName="kubernetes.io/secret/fb047779-ad10-4f7e-a45a-4a154ab60ad8-image-registry-private-configuration" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054088 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="16a02457-151b-4e96-8118-e7695190b9be" volumeName="kubernetes.io/projected/16a02457-151b-4e96-8118-e7695190b9be-kube-api-access-dc5nm" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054101 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5133b471-c5f8-4b35-b06c-1d8f429c3ba2" volumeName="kubernetes.io/configmap/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-certificates" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054117 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5133b471-c5f8-4b35-b06c-1d8f429c3ba2" volumeName="kubernetes.io/configmap/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-trusted-ca" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054131 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52a4a41e-8b25-4e5f-86ad-597dcd08cbd3" volumeName="kubernetes.io/configmap/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-trusted-ca-bundle" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054144 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52a4a41e-8b25-4e5f-86ad-597dcd08cbd3" volumeName="kubernetes.io/configmap/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-service-ca-bundle" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054158 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="85617466-6b8e-45d4-a45e-e9650078c0be" volumeName="kubernetes.io/projected/85617466-6b8e-45d4-a45e-e9650078c0be-kube-api-access-9cgzb" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054171 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="49b68250-16ee-42f1-9730-e4d326c96cd8" volumeName="kubernetes.io/configmap/49b68250-16ee-42f1-9730-e4d326c96cd8-konnectivity-ca" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054183 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="38b6bfaf-2fb2-455b-9e7d-32586b3f216f" volumeName="kubernetes.io/empty-dir/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-tmp-dir" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054195 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f0d51a79-823b-4059-94c3-0535fc882a44" volumeName="kubernetes.io/configmap/f0d51a79-823b-4059-94c3-0535fc882a44-telemetry-config" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054210 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb047779-ad10-4f7e-a45a-4a154ab60ad8" volumeName="kubernetes.io/secret/fb047779-ad10-4f7e-a45a-4a154ab60ad8-installation-pull-secrets" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054222 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fdddab43-acaf-4153-bb52-3e60044a35dc" volumeName="kubernetes.io/configmap/fdddab43-acaf-4153-bb52-3e60044a35dc-trusted-ca" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054234 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fdddab43-acaf-4153-bb52-3e60044a35dc" volumeName="kubernetes.io/secret/fdddab43-acaf-4153-bb52-3e60044a35dc-serving-cert" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054246 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23d95081-1188-4002-997e-65ca57111fab" volumeName="kubernetes.io/secret/23d95081-1188-4002-997e-65ca57111fab-ovn-node-metrics-cert" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054259 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3162c3d2-e86f-4ae1-8196-0414509fae9e" volumeName="kubernetes.io/configmap/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-daemon-config" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054272 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="38b6bfaf-2fb2-455b-9e7d-32586b3f216f" volumeName="kubernetes.io/configmap/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-config-volume" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054286 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3e77ccc1-14a3-4f85-a214-854bd392733a" volumeName="kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-stats-auth" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054307 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52a4a41e-8b25-4e5f-86ad-597dcd08cbd3" volumeName="kubernetes.io/projected/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-kube-api-access-d5fjr" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054320 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52ca5902-69f0-462f-ae35-cbbc7f378345" volumeName="kubernetes.io/secret/52ca5902-69f0-462f-ae35-cbbc7f378345-serving-cert" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054332 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="66992144-51dd-4c1d-95c8-63862bca97cc" volumeName="kubernetes.io/configmap/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-config" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054345 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4c9c543-f712-4a48-949f-d3f3bc8f77e1" volumeName="kubernetes.io/empty-dir/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-tmp" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054357 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0e13113b-b4d2-42d7-9e1c-642a790bcd55" volumeName="kubernetes.io/projected/0e13113b-b4d2-42d7-9e1c-642a790bcd55-kube-api-access-j8ksd" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054372 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1734d386-0cb1-4002-a954-b76b3cf62eaf" volumeName="kubernetes.io/projected/1734d386-0cb1-4002-a954-b76b3cf62eaf-kube-api-access-j4v4t" seLinuxMountContext="" Oct 30 12:51:41.054376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054385 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5133b471-c5f8-4b35-b06c-1d8f429c3ba2" volumeName="kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-kube-api-access-gjl5n" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054398 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5cd3297b-e3fe-40e0-be87-88a67236d9f0" volumeName="kubernetes.io/empty-dir/5cd3297b-e3fe-40e0-be87-88a67236d9f0-data-volume" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054411 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="75bd9ac5-49ff-4c58-85c7-79b181150555" volumeName="kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054423 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb047779-ad10-4f7e-a45a-4a154ab60ad8" volumeName="kubernetes.io/configmap/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-certificates" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054438 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb047779-ad10-4f7e-a45a-4a154ab60ad8" volumeName="kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-bound-sa-token" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054451 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fdddab43-acaf-4153-bb52-3e60044a35dc" volumeName="kubernetes.io/configmap/fdddab43-acaf-4153-bb52-3e60044a35dc-config" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054463 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0d6a61ca-a15f-4742-8e81-6057c23e51c8" volumeName="kubernetes.io/configmap/0d6a61ca-a15f-4742-8e81-6057c23e51c8-iptables-alerter-script" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054488 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5133b471-c5f8-4b35-b06c-1d8f429c3ba2" volumeName="kubernetes.io/empty-dir/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-ca-trust-extracted" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054500 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52ca5902-69f0-462f-ae35-cbbc7f378345" volumeName="kubernetes.io/projected/52ca5902-69f0-462f-ae35-cbbc7f378345-kube-api-access-n7r5b" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054513 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7d93802e-f47d-4319-b308-04fe84901551" volumeName="kubernetes.io/projected/7d93802e-f47d-4319-b308-04fe84901551-kube-api-access-64858" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054527 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a5428d1a-8c8b-41ce-ad97-a729ec7a2e43" volumeName="kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054539 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4ca5142-5b8a-4406-9ed0-06f9edea89bc" volumeName="kubernetes.io/secret/d4ca5142-5b8a-4406-9ed0-06f9edea89bc-cert" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054549 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b1634e9-3028-4b6a-b651-9155018b3c5a" volumeName="kubernetes.io/projected/0b1634e9-3028-4b6a-b651-9155018b3c5a-kube-api-access-997qf" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054562 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23d95081-1188-4002-997e-65ca57111fab" volumeName="kubernetes.io/configmap/23d95081-1188-4002-997e-65ca57111fab-ovnkube-script-lib" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054572 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2a12192e-f0c8-4354-b9fa-1b65b04ebc19" volumeName="kubernetes.io/projected/2a12192e-f0c8-4354-b9fa-1b65b04ebc19-kube-api-access-mld9s" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054582 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3e77ccc1-14a3-4f85-a214-854bd392733a" volumeName="kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-default-certificate" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054593 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5133b471-c5f8-4b35-b06c-1d8f429c3ba2" volumeName="kubernetes.io/secret/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-image-registry-private-configuration" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054604 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4ca5142-5b8a-4406-9ed0-06f9edea89bc" volumeName="kubernetes.io/projected/d4ca5142-5b8a-4406-9ed0-06f9edea89bc-kube-api-access-5h7bv" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054613 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e9446374-a7ea-4f95-baa4-d6dad6805b65" volumeName="kubernetes.io/configmap/e9446374-a7ea-4f95-baa4-d6dad6805b65-cni-sysctl-allowlist" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054624 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e9446374-a7ea-4f95-baa4-d6dad6805b65" volumeName="kubernetes.io/configmap/e9446374-a7ea-4f95-baa4-d6dad6805b65-whereabouts-flatfile-configmap" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054634 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d" volumeName="kubernetes.io/empty-dir/7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d-tmp-dir" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054645 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fdddab43-acaf-4153-bb52-3e60044a35dc" volumeName="kubernetes.io/projected/fdddab43-acaf-4153-bb52-3e60044a35dc-kube-api-access-c59v8" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054666 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52a4a41e-8b25-4e5f-86ad-597dcd08cbd3" volumeName="kubernetes.io/empty-dir/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-archives-path" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054680 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f0d51a79-823b-4059-94c3-0535fc882a44" volumeName="kubernetes.io/projected/f0d51a79-823b-4059-94c3-0535fc882a44-kube-api-access-scm4l" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054691 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3162c3d2-e86f-4ae1-8196-0414509fae9e" volumeName="kubernetes.io/configmap/3162c3d2-e86f-4ae1-8196-0414509fae9e-cni-binary-copy" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054704 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462" volumeName="kubernetes.io/configmap/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-service-ca-bundle" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054716 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="49b68250-16ee-42f1-9730-e4d326c96cd8" volumeName="kubernetes.io/secret/49b68250-16ee-42f1-9730-e4d326c96cd8-agent-certs" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054730 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3162c3d2-e86f-4ae1-8196-0414509fae9e" volumeName="kubernetes.io/projected/3162c3d2-e86f-4ae1-8196-0414509fae9e-kube-api-access-7vkl7" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054743 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9951ec02-ee4f-488c-8cab-3985569328f8" volumeName="kubernetes.io/secret/9951ec02-ee4f-488c-8cab-3985569328f8-serving-cert" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054756 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e9446374-a7ea-4f95-baa4-d6dad6805b65" volumeName="kubernetes.io/configmap/e9446374-a7ea-4f95-baa4-d6dad6805b65-cni-binary-copy" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054769 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5133b471-c5f8-4b35-b06c-1d8f429c3ba2" volumeName="kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-bound-sa-token" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054783 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9951ec02-ee4f-488c-8cab-3985569328f8" volumeName="kubernetes.io/configmap/9951ec02-ee4f-488c-8cab-3985569328f8-config" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054796 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9951ec02-ee4f-488c-8cab-3985569328f8" volumeName="kubernetes.io/projected/9951ec02-ee4f-488c-8cab-3985569328f8-kube-api-access-htqvt" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054808 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4c9c543-f712-4a48-949f-d3f3bc8f77e1" volumeName="kubernetes.io/projected/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-kube-api-access-pmqmx" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054821 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb047779-ad10-4f7e-a45a-4a154ab60ad8" volumeName="kubernetes.io/empty-dir/fb047779-ad10-4f7e-a45a-4a154ab60ad8-ca-trust-extracted" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054835 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23d95081-1188-4002-997e-65ca57111fab" volumeName="kubernetes.io/configmap/23d95081-1188-4002-997e-65ca57111fab-env-overrides" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054848 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0d6a61ca-a15f-4742-8e81-6057c23e51c8" volumeName="kubernetes.io/projected/0d6a61ca-a15f-4742-8e81-6057c23e51c8-kube-api-access-5djfp" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054864 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="109e0336-ecdb-4278-b5d3-72c7bdb80276" volumeName="kubernetes.io/configmap/109e0336-ecdb-4278-b5d3-72c7bdb80276-nginx-conf" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054907 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="66992144-51dd-4c1d-95c8-63862bca97cc" volumeName="kubernetes.io/projected/66992144-51dd-4c1d-95c8-63862bca97cc-kube-api-access-mdmq4" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054922 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d" volumeName="kubernetes.io/projected/7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d-kube-api-access-mvwvc" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054934 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e9446374-a7ea-4f95-baa4-d6dad6805b65" volumeName="kubernetes.io/projected/e9446374-a7ea-4f95-baa4-d6dad6805b65-kube-api-access-wd65z" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054948 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52ca5902-69f0-462f-ae35-cbbc7f378345" volumeName="kubernetes.io/configmap/52ca5902-69f0-462f-ae35-cbbc7f378345-config" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054960 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5cd3297b-e3fe-40e0-be87-88a67236d9f0" volumeName="kubernetes.io/projected/5cd3297b-e3fe-40e0-be87-88a67236d9f0-kube-api-access-fhns5" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054974 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462" volumeName="kubernetes.io/empty-dir/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-snapshots" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.054987 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462" volumeName="kubernetes.io/projected/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-kube-api-access-xg5w5" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.055000 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb047779-ad10-4f7e-a45a-4a154ab60ad8" volumeName="kubernetes.io/configmap/fb047779-ad10-4f7e-a45a-4a154ab60ad8-trusted-ca" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.055013 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb047779-ad10-4f7e-a45a-4a154ab60ad8" volumeName="kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-kube-api-access-rk88t" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.055027 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a8f7b67-6d17-4f5e-a79b-52481ea0fd48" volumeName="kubernetes.io/projected/1a8f7b67-6d17-4f5e-a79b-52481ea0fd48-kube-api-access-srvwc" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.055039 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="23d95081-1188-4002-997e-65ca57111fab" volumeName="kubernetes.io/configmap/23d95081-1188-4002-997e-65ca57111fab-ovnkube-config" seLinuxMountContext="" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.055052 5029 reconstruct.go:97] "Volume reconstruction finished" Oct 30 12:51:41.055659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.055060 5029 reconciler.go:29] "Reconciler: start to sync state" Oct 30 12:51:41.071773 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.071741 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/1.log" Oct 30 12:51:41.072636 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.072613 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/1.log" Oct 30 12:51:41.079443 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.079414 5029 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 12:51:41.081059 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.081037 5029 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 12:51:41.081059 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.081060 5029 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 12:51:41.081448 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.081426 5029 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 12:51:41.081508 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:41.081491 5029 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Oct 30 12:51:41.092839 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.092809 5029 generic.go:356] "Generic (PLEG): container finished" podID="e9446374-a7ea-4f95-baa4-d6dad6805b65" containerID="abd64f2cecd97ddaacd2d62288e00b695e39f913c11265a59a417c92088604dc" exitCode=0 Oct 30 12:51:41.092839 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.092836 5029 generic.go:356] "Generic (PLEG): container finished" podID="e9446374-a7ea-4f95-baa4-d6dad6805b65" containerID="a22a214c42f5c88e59e2e505914fc2acf44e9f215a6ab18235d109ec0a5ef446" exitCode=0 Oct 30 12:51:41.092839 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.092845 5029 generic.go:356] "Generic (PLEG): container finished" podID="e9446374-a7ea-4f95-baa4-d6dad6805b65" containerID="3e594a5b0da345b7bbaf500a20195a2b48901c88e697feb6d5b61b3c97671274" exitCode=0 Oct 30 12:51:41.092839 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.092854 5029 generic.go:356] "Generic (PLEG): container finished" podID="e9446374-a7ea-4f95-baa4-d6dad6805b65" containerID="0bb3feb1bc32a7dc61595f07d20cef372de3574fff34c2cbdf0be697fb45c4f7" exitCode=0 Oct 30 12:51:41.093124 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.092861 5029 generic.go:356] "Generic (PLEG): container finished" podID="e9446374-a7ea-4f95-baa4-d6dad6805b65" containerID="846cde979c42d7b1e3ca8a1d674e0dacb10ed1ae6d8cd660ca4e2ff337e16b73" exitCode=0 Oct 30 12:51:41.093124 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.092868 5029 generic.go:356] "Generic (PLEG): container finished" podID="e9446374-a7ea-4f95-baa4-d6dad6805b65" containerID="52a11e7920fe35baa1e8429608f4594144aaad9e47e07ab210c67fc3131b470e" exitCode=0 Oct 30 12:51:41.100309 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.100287 5029 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Oct 30 12:51:41.113819 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.113791 5029 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 12:51:41.113923 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.113910 5029 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 12:51:41.113988 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.113937 5029 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 12:51:41.114154 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.114138 5029 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 12:51:41.114423 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.114405 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/1.log" Oct 30 12:51:41.114469 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.114438 5029 generic.go:356] "Generic (PLEG): container finished" podID="fdddab43-acaf-4153-bb52-3e60044a35dc" containerID="72e9b907fdc4c8a3603e98b9c0298a772b043c460a2a2a72a020710cd8e9eeeb" exitCode=255 Oct 30 12:51:41.136821 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.136793 5029 generic.go:356] "Generic (PLEG): container finished" podID="3f5d6256264944b543b87edf9d8fc228" containerID="655bad5f6df15aa40346ac1df3071fd2e74d1b2b9661432c9fc3e387afe23664" exitCode=0 Oct 30 12:51:41.152151 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.152134 5029 manager.go:324] Recovery completed Oct 30 12:51:41.173484 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.173427 5029 cpu_manager.go:226] "Starting" policy="none" Oct 30 12:51:41.173484 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.173448 5029 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 12:51:41.173484 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.173469 5029 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 12:51:41.173668 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.173642 5029 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Oct 30 12:51:41.173712 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.173657 5029 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Oct 30 12:51:41.173712 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.173681 5029 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Oct 30 12:51:41.173712 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.173689 5029 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Oct 30 12:51:41.173712 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.173696 5029 policy_none.go:50] "Start" Oct 30 12:51:41.173865 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.173715 5029 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 12:51:41.173865 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.173728 5029 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 12:51:41.174110 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.174089 5029 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Oct 30 12:51:41.174168 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.174113 5029 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Oct 30 12:51:41.174168 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.174123 5029 policy_none.go:44] "Start" Oct 30 12:51:41.178846 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.178820 5029 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 12:51:41.181376 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.181361 5029 manager.go:341] "Starting Device Plugin manager" Oct 30 12:51:41.181618 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:41.181605 5029 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Oct 30 12:51:41.181915 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:41.181386 5029 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 12:51:41.181955 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.181919 5029 server.go:87] "Starting device plugin registration server" Oct 30 12:51:41.182188 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.182176 5029 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 12:51:41.182228 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.182190 5029 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 12:51:41.182314 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.182299 5029 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 12:51:41.182434 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.182398 5029 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 12:51:41.182532 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.182442 5029 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 12:51:41.182532 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.182448 5029 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 12:51:41.182628 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.182496 5029 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T12:51:41.182429163Z","UUID":"0eb09250-9459-4a95-8abb-e2f264cf7c90","Handler":null,"Name":"","Endpoint":""} Oct 30 12:51:41.183858 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:41.183826 5029 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 12:51:41.183966 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.183834 5029 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 12:51:41.183966 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.183939 5029 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 12:51:41.282709 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.282680 5029 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 12:51:41.284807 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.284780 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasSufficientMemory" Oct 30 12:51:41.284807 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.284810 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasNoDiskPressure" Oct 30 12:51:41.284921 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.284820 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-147.ec2.internal" event="NodeHasSufficientPID" Oct 30 12:51:41.284921 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.284861 5029 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-141-147.ec2.internal" Oct 30 12:51:41.324655 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.324628 5029 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-141-147.ec2.internal" Oct 30 12:51:41.324780 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.324697 5029 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-141-147.ec2.internal" Oct 30 12:51:41.381927 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.381882 5029 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-147.ec2.internal"] Oct 30 12:51:41.382619 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.382602 5029 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" Oct 30 12:51:41.382704 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.382670 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-147.ec2.internal" event={"ID":"7d0024b3080a170827a29027ec64e58f","Type":"ContainerStarted","Data":"8f91a6e7c2cea557ea711ab965883c40d084b0a5088b76cbcd22b035fc618fde"} Oct 30 12:51:41.382745 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.382721 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-147.ec2.internal" event={"ID":"7d0024b3080a170827a29027ec64e58f","Type":"ContainerStarted","Data":"31b370d6fd236b02a8a8fbd660474aaa1fa31699add3ccee185e54d698bddad0"} Oct 30 12:51:41.382784 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.382770 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" event={"ID":"3f5d6256264944b543b87edf9d8fc228","Type":"ContainerStarted","Data":"134f2a06d7e58292517228c2edb72b425436a1c722c3322122e6be671d1717b7"} Oct 30 12:51:41.382818 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.382789 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" event={"ID":"3f5d6256264944b543b87edf9d8fc228","Type":"ContainerDied","Data":"655bad5f6df15aa40346ac1df3071fd2e74d1b2b9661432c9fc3e387afe23664"} Oct 30 12:51:41.382818 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.382808 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" event={"ID":"3f5d6256264944b543b87edf9d8fc228","Type":"ContainerStarted","Data":"c0b42f36f921b49fec4166c8f47d231869253e78239ef978d91924ed1d90d1ae"} Oct 30 12:51:41.382881 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.382822 5029 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-147.ec2.internal" Oct 30 12:51:41.411728 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.411701 5029 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 12:51:41.411821 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:41.411746 5029 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-141-147.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-147.ec2.internal" Oct 30 12:51:41.413352 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.413338 5029 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 12:51:41.413394 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:41.413374 5029 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" Oct 30 12:51:41.501832 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.501760 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3f5d6256264944b543b87edf9d8fc228-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal\" (UID: \"3f5d6256264944b543b87edf9d8fc228\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" Oct 30 12:51:41.501832 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.501790 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f5d6256264944b543b87edf9d8fc228-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal\" (UID: \"3f5d6256264944b543b87edf9d8fc228\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" Oct 30 12:51:41.501832 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.501809 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7d0024b3080a170827a29027ec64e58f-config\") pod \"kube-apiserver-proxy-ip-10-0-141-147.ec2.internal\" (UID: \"7d0024b3080a170827a29027ec64e58f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-147.ec2.internal" Oct 30 12:51:41.602293 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.602258 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3f5d6256264944b543b87edf9d8fc228-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal\" (UID: \"3f5d6256264944b543b87edf9d8fc228\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" Oct 30 12:51:41.602293 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.602292 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f5d6256264944b543b87edf9d8fc228-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal\" (UID: \"3f5d6256264944b543b87edf9d8fc228\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" Oct 30 12:51:41.602483 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.602314 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7d0024b3080a170827a29027ec64e58f-config\") pod \"kube-apiserver-proxy-ip-10-0-141-147.ec2.internal\" (UID: \"7d0024b3080a170827a29027ec64e58f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-147.ec2.internal" Oct 30 12:51:41.602483 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.602365 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7d0024b3080a170827a29027ec64e58f-config\") pod \"kube-apiserver-proxy-ip-10-0-141-147.ec2.internal\" (UID: \"7d0024b3080a170827a29027ec64e58f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-147.ec2.internal" Oct 30 12:51:41.602483 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.602378 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3f5d6256264944b543b87edf9d8fc228-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal\" (UID: \"3f5d6256264944b543b87edf9d8fc228\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" Oct 30 12:51:41.602483 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:41.602365 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3f5d6256264944b543b87edf9d8fc228-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal\" (UID: \"3f5d6256264944b543b87edf9d8fc228\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" Oct 30 12:51:42.037563 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.037527 5029 apiserver.go:52] "Watching apiserver" Oct 30 12:51:42.141154 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.141124 5029 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" Oct 30 12:51:42.141305 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.141193 5029 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-147.ec2.internal" Oct 30 12:51:42.163036 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.163011 5029 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 12:51:42.163179 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.163048 5029 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 12:51:42.163179 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:42.163107 5029 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-141-147.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-147.ec2.internal" Oct 30 12:51:42.163793 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.163765 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-nzp7p","openshift-kube-storage-version-migrator/migrator-66df5d496b-q68tl","openshift-ingress-canary/ingress-canary-fnmsk","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr","kube-system/kas-connection-checker-685895c7c9-wsfvm","kube-system/konnectivity-agent-vjn55","openshift-kube-storage-version-migrator/migrator-66df5d496b-pjfzl","openshift-network-diagnostics/network-check-source-6f78d59f99-xsw6d","openshift-image-registry/image-registry-5cd77977d7-wrk42","openshift-network-console/networking-console-plugin-86c478bb88-49h5k","kube-system/kas-connection-checker-685895c7c9-nprbr","kube-system/global-pull-secret-syncer-lf7b2","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-bzh2m","openshift-console-operator/console-operator-7cd56b8dbc-nbwj8","openshift-insights/insights-runtime-extractor-l7z89","openshift-insights/periodic-gathering-vs88s-f66ls","openshift-dns/node-resolver-759xk","kube-system/kube-apiserver-proxy-ip-10-0-141-147.ec2.internal","openshift-ingress/router-default-767cffcd47-qx67t","openshift-network-operator/iptables-alerter-2cvn5","openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp","openshift-multus/multus-additional-cni-plugins-s2xpm","openshift-ovn-kubernetes/ovnkube-node-cm78w","openshift-cluster-node-tuning-operator/tuned-qwj8g","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal","openshift-multus/network-metrics-daemon-24p2b","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr","openshift-image-registry/image-registry-5cfd489894-v5wlw","openshift-network-diagnostics/network-check-target-d6rt6","openshift-insights/insights-operator-7c4986d449-bbb6d","openshift-dns/dns-default-cd6w9","openshift-image-registry/node-ca-vsf9w","openshift-multus/multus-hjpbf","openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn"] Oct 30 12:51:42.165036 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.165017 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:42.166919 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.166886 5029 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 12:51:42.167015 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:42.166960 5029 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-147.ec2.internal" Oct 30 12:51:42.167435 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.167415 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:42.167737 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.167718 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:42.168924 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.168909 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:42.172702 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.172684 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-49h5k" Oct 30 12:51:42.177934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.177917 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:42.178531 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.178513 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:42.179135 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.179120 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:42.181000 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.180981 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\"" Oct 30 12:51:42.181119 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.181104 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:42.181249 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.181220 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\"" Oct 30 12:51:42.181249 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.181239 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Oct 30 12:51:42.181390 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.181253 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Oct 30 12:51:42.181390 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.181305 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Oct 30 12:51:42.181496 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.181433 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Oct 30 12:51:42.181553 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.181514 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Oct 30 12:51:42.181553 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.181535 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Oct 30 12:51:42.181553 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.181544 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Oct 30 12:51:42.181727 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.181611 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-kzgpw\"" Oct 30 12:51:42.181727 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.181634 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Oct 30 12:51:42.182022 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.182004 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-v2wwb\"" Oct 30 12:51:42.183181 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.183164 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Oct 30 12:51:42.183430 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.183413 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Oct 30 12:51:42.183543 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.183523 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Oct 30 12:51:42.183592 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.183581 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\"" Oct 30 12:51:42.235535 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.235513 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Oct 30 12:51:42.236477 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.236457 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 12:51:42.236638 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.236617 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Oct 30 12:51:42.236710 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.236664 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 12:51:42.236837 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.236817 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 12:51:42.236992 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.236971 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 12:51:42.237292 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.237274 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 12:51:42.237371 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.237318 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Oct 30 12:51:42.237371 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.237362 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 12:51:42.237648 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.237624 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 12:51:42.240289 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240250 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 12:51:42.240392 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240289 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 12:51:42.240392 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240310 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 12:51:42.240392 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240318 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Oct 30 12:51:42.240392 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240311 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Oct 30 12:51:42.240392 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240355 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Oct 30 12:51:42.240392 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240387 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 12:51:42.240654 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240419 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Oct 30 12:51:42.240654 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240440 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 12:51:42.240654 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240453 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 12:51:42.240654 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240453 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Oct 30 12:51:42.240654 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240442 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 12:51:42.240654 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240536 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Oct 30 12:51:42.240654 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240559 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 12:51:42.240654 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240583 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Oct 30 12:51:42.240654 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240606 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Oct 30 12:51:42.240654 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240559 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 12:51:42.240654 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240641 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 12:51:42.240654 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240648 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 12:51:42.241005 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240672 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 12:51:42.241005 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240680 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 12:51:42.241005 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240698 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 12:51:42.241005 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240721 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 12:51:42.241005 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240737 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 12:51:42.241005 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.240773 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 12:51:42.241285 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.241045 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Oct 30 12:51:42.241285 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.241109 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 12:51:42.241285 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.241184 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 12:51:42.241285 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.241196 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 12:51:42.241285 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.241253 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Oct 30 12:51:42.241285 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.241260 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 12:51:42.241285 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.241272 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 12:51:42.241285 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.241278 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 12:51:42.241648 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.241313 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Oct 30 12:51:42.241648 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.241322 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 12:51:42.241648 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.241346 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 12:51:42.241648 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.241323 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 12:51:42.241648 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.241451 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 12:51:42.241648 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.241580 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Oct 30 12:51:42.241862 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.241738 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Oct 30 12:51:42.248119 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.248104 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Oct 30 12:51:42.252124 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.252110 5029 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 12:51:42.282243 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.282225 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Oct 30 12:51:42.282325 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.282288 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Oct 30 12:51:42.282325 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.282303 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Oct 30 12:51:42.282433 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.282372 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Oct 30 12:51:42.282479 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.282459 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 12:51:42.282582 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.282564 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 12:51:42.282693 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.282570 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-48745\"" Oct 30 12:51:42.282693 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.282636 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 12:51:42.282774 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.282714 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 12:51:42.282774 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.282743 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-bhhxq\"" Oct 30 12:51:42.282841 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.282778 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Oct 30 12:51:42.282841 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.282784 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Oct 30 12:51:42.282906 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.282882 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-k7gdg\"" Oct 30 12:51:42.283025 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283013 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Oct 30 12:51:42.283090 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283040 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-sgjxx\"" Oct 30 12:51:42.283090 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283056 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-9lc5z\"" Oct 30 12:51:42.283193 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283179 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-68fxr\"" Oct 30 12:51:42.283259 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283247 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qjjfj\"" Oct 30 12:51:42.283308 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283279 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xj7mb\"" Oct 30 12:51:42.283308 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283268 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-2x7bq\"" Oct 30 12:51:42.283308 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283289 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ktwvb\"" Oct 30 12:51:42.283415 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283289 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hszmd\"" Oct 30 12:51:42.283415 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283388 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Oct 30 12:51:42.283415 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283397 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9ppdm\"" Oct 30 12:51:42.283552 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283441 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6jkd8\"" Oct 30 12:51:42.283552 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283456 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Oct 30 12:51:42.283552 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283477 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-xd4gf\"" Oct 30 12:51:42.283552 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283489 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 12:51:42.283552 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283442 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-p6djf\"" Oct 30 12:51:42.283552 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283491 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-87w8v\"" Oct 30 12:51:42.283552 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283507 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7j54b\"" Oct 30 12:51:42.283552 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283484 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-wqgtx\"" Oct 30 12:51:42.283552 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283446 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-zxmvc\"" Oct 30 12:51:42.283552 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283527 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cglsl\"" Oct 30 12:51:42.283552 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283541 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 12:51:42.283552 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283542 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-m2lrf\"" Oct 30 12:51:42.283905 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283542 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-5t8jx\"" Oct 30 12:51:42.283905 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283547 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-v4xk8\"" Oct 30 12:51:42.283905 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283584 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 12:51:42.283905 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283618 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-sfhnn\"" Oct 30 12:51:42.283905 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283655 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-45hdr\"" Oct 30 12:51:42.283905 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.283701 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 12:51:42.305994 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.305929 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:42.305994 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.305968 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-ca-trust-extracted\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:42.305994 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.305985 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4v4t\" (UniqueName: \"kubernetes.io/projected/1734d386-0cb1-4002-a954-b76b3cf62eaf-kube-api-access-j4v4t\") pod \"kas-connection-checker-685895c7c9-wsfvm\" (UID: \"1734d386-0cb1-4002-a954-b76b3cf62eaf\") " pod="kube-system/kas-connection-checker-685895c7c9-wsfvm" Oct 30 12:51:42.306202 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306000 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:42.306202 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306082 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/23d95081-1188-4002-997e-65ca57111fab-ovnkube-config\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.306202 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306108 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-daemon-config\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.306202 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306194 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-ca-trust-extracted\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:42.306443 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306236 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmqmx\" (UniqueName: \"kubernetes.io/projected/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-kube-api-access-pmqmx\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.306443 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306266 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/49b68250-16ee-42f1-9730-e4d326c96cd8-konnectivity-ca\") pod \"konnectivity-agent-vjn55\" (UID: \"49b68250-16ee-42f1-9730-e4d326c96cd8\") " pod="kube-system/konnectivity-agent-vjn55" Oct 30 12:51:42.306443 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306289 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f0d51a79-823b-4059-94c3-0535fc882a44-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:42.306443 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306313 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-registration-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:42.306443 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306428 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scm4l\" (UniqueName: \"kubernetes.io/projected/f0d51a79-823b-4059-94c3-0535fc882a44-kube-api-access-scm4l\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:42.306690 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306453 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-daemon-config\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.306690 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306458 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/23d95081-1188-4002-997e-65ca57111fab-ovnkube-config\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.306690 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306456 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-archives-path\") pod \"periodic-gathering-vs88s-f66ls\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:51:42.306690 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306508 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-run-netns\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.306690 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306533 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-system-cni-dir\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.306690 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306542 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-archives-path\") pod \"periodic-gathering-vs88s-f66ls\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:51:42.306690 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306558 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-cnibin\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.306690 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306563 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f0d51a79-823b-4059-94c3-0535fc882a44-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:42.306690 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306563 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/49b68250-16ee-42f1-9730-e4d326c96cd8-konnectivity-ca\") pod \"konnectivity-agent-vjn55\" (UID: \"49b68250-16ee-42f1-9730-e4d326c96cd8\") " pod="kube-system/konnectivity-agent-vjn55" Oct 30 12:51:42.306690 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306587 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-sys\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.306690 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306613 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/49b68250-16ee-42f1-9730-e4d326c96cd8-agent-certs\") pod \"konnectivity-agent-vjn55\" (UID: \"49b68250-16ee-42f1-9730-e4d326c96cd8\") " pod="kube-system/konnectivity-agent-vjn55" Oct 30 12:51:42.306690 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306638 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg5w5\" (UniqueName: \"kubernetes.io/projected/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-kube-api-access-xg5w5\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:42.306690 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306682 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-run-openvswitch\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.307159 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306791 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-run-multus-certs\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.307159 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306850 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-certificates\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:42.307159 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306856 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/49b68250-16ee-42f1-9730-e4d326c96cd8-agent-certs\") pod \"konnectivity-agent-vjn55\" (UID: \"49b68250-16ee-42f1-9730-e4d326c96cd8\") " pod="kube-system/konnectivity-agent-vjn55" Oct 30 12:51:42.307159 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306876 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fb047779-ad10-4f7e-a45a-4a154ab60ad8-installation-pull-secrets\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:42.307159 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306909 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-metrics-tls\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:42.307159 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306940 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhns5\" (UniqueName: \"kubernetes.io/projected/5cd3297b-e3fe-40e0-be87-88a67236d9f0-kube-api-access-fhns5\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:42.307159 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.306968 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e77ccc1-14a3-4f85-a214-854bd392733a-service-ca-bundle\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:42.307159 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307000 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fb047779-ad10-4f7e-a45a-4a154ab60ad8-ca-trust-extracted\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:42.307159 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307002 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-certificates\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:42.307159 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307019 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fb047779-ad10-4f7e-a45a-4a154ab60ad8-installation-pull-secrets\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:42.307159 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307043 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/75bd9ac5-49ff-4c58-85c7-79b181150555-kubelet-config\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:42.307159 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307055 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fb047779-ad10-4f7e-a45a-4a154ab60ad8-ca-trust-extracted\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:42.307159 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307099 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-etc-openvswitch\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.307159 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307141 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mld9s\" (UniqueName: \"kubernetes.io/projected/2a12192e-f0c8-4354-b9fa-1b65b04ebc19-kube-api-access-mld9s\") pod \"node-ca-vsf9w\" (UID: \"2a12192e-f0c8-4354-b9fa-1b65b04ebc19\") " pod="openshift-image-registry/node-ca-vsf9w" Oct 30 12:51:42.307799 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307175 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-cni-dir\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.307799 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307206 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-conf-dir\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.307799 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307234 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/52ca5902-69f0-462f-ae35-cbbc7f378345-config\") pod \"service-ca-operator-544d749c8c-fk7rp\" (UID: \"52ca5902-69f0-462f-ae35-cbbc7f378345\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp" Oct 30 12:51:42.307799 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307301 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64858\" (UniqueName: \"kubernetes.io/projected/7d93802e-f47d-4319-b308-04fe84901551-kube-api-access-64858\") pod \"kas-connection-checker-685895c7c9-nzp7p\" (UID: \"7d93802e-f47d-4319-b308-04fe84901551\") " pod="kube-system/kas-connection-checker-685895c7c9-nzp7p" Oct 30 12:51:42.307799 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307367 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/52ca5902-69f0-462f-ae35-cbbc7f378345-config\") pod \"service-ca-operator-544d749c8c-fk7rp\" (UID: \"52ca5902-69f0-462f-ae35-cbbc7f378345\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp" Oct 30 12:51:42.307799 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307379 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:42.307799 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307398 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-config-volume\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:42.307799 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307414 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-lib-modules\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.307799 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307439 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/109e0336-ecdb-4278-b5d3-72c7bdb80276-nginx-conf\") pod \"networking-console-plugin-86c478bb88-49h5k\" (UID: \"109e0336-ecdb-4278-b5d3-72c7bdb80276\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-49h5k" Oct 30 12:51:42.307799 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307539 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-var-lib-openvswitch\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.307799 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307555 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-node-log\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.307799 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307562 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/75bd9ac5-49ff-4c58-85c7-79b181150555-original-pull-secret\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:42.307799 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307570 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-var-lib-cni-bin\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.307799 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307594 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-run\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.307799 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307616 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:42.307799 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307613 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-config-volume\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:42.307799 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307651 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/109e0336-ecdb-4278-b5d3-72c7bdb80276-nginx-conf\") pod \"networking-console-plugin-86c478bb88-49h5k\" (UID: \"109e0336-ecdb-4278-b5d3-72c7bdb80276\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-49h5k" Oct 30 12:51:42.307799 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307658 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-certificates\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:42.307799 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307693 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srvwc\" (UniqueName: \"kubernetes.io/projected/1a8f7b67-6d17-4f5e-a79b-52481ea0fd48-kube-api-access-srvwc\") pod \"volume-data-source-validator-58c9f5f46d-bzh2m\" (UID: \"1a8f7b67-6d17-4f5e-a79b-52481ea0fd48\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-bzh2m" Oct 30 12:51:42.307799 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307722 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-etc-selinux\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:42.307799 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307794 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-certificates\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307854 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-tmp\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307878 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fb047779-ad10-4f7e-a45a-4a154ab60ad8-image-registry-private-configuration\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307901 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cgzb\" (UniqueName: \"kubernetes.io/projected/85617466-6b8e-45d4-a45e-e9650078c0be-kube-api-access-9cgzb\") pod \"migrator-66df5d496b-pjfzl\" (UID: \"85617466-6b8e-45d4-a45e-e9650078c0be\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-pjfzl" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307914 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2a12192e-f0c8-4354-b9fa-1b65b04ebc19-serviceca\") pod \"node-ca-vsf9w\" (UID: \"2a12192e-f0c8-4354-b9fa-1b65b04ebc19\") " pod="openshift-image-registry/node-ca-vsf9w" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.307929 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308007 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-tmp\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308039 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-trusted-ca-bundle\") pod \"periodic-gathering-vs88s-f66ls\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308089 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308117 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-cni-netd\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308115 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fb047779-ad10-4f7e-a45a-4a154ab60ad8-image-registry-private-configuration\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308140 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2a12192e-f0c8-4354-b9fa-1b65b04ebc19-serviceca\") pod \"node-ca-vsf9w\" (UID: \"2a12192e-f0c8-4354-b9fa-1b65b04ebc19\") " pod="openshift-image-registry/node-ca-vsf9w" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308151 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/23d95081-1188-4002-997e-65ca57111fab-ovn-node-metrics-cert\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308173 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3162c3d2-e86f-4ae1-8196-0414509fae9e-cni-binary-copy\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308213 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308247 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-etc-kubernetes\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308279 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vkl7\" (UniqueName: \"kubernetes.io/projected/3162c3d2-e86f-4ae1-8196-0414509fae9e-kube-api-access-7vkl7\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308286 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/23d95081-1188-4002-997e-65ca57111fab-ovn-node-metrics-cert\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308322 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-sys-fs\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308362 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-trusted-ca-bundle\") pod \"periodic-gathering-vs88s-f66ls\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308405 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3162c3d2-e86f-4ae1-8196-0414509fae9e-cni-binary-copy\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308423 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-snapshots\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308469 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-systemd-units\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308501 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w928x\" (UniqueName: \"kubernetes.io/projected/6d7c203f-f696-471f-baf8-32c0ad647baf-kube-api-access-w928x\") pod \"migrator-66df5d496b-q68tl\" (UID: \"6d7c203f-f696-471f-baf8-32c0ad647baf\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-q68tl" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308505 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-snapshots\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308525 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d-tmp-dir\") pod \"node-resolver-759xk\" (UID: \"7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d\") " pod="openshift-dns/node-resolver-759xk" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308564 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5cd3297b-e3fe-40e0-be87-88a67236d9f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308587 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9951ec02-ee4f-488c-8cab-3985569328f8-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr\" (UID: \"9951ec02-ee4f-488c-8cab-3985569328f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308711 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9951ec02-ee4f-488c-8cab-3985569328f8-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr\" (UID: \"9951ec02-ee4f-488c-8cab-3985569328f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr" Oct 30 12:51:42.308717 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308725 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d-tmp-dir\") pod \"node-resolver-759xk\" (UID: \"7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d\") " pod="openshift-dns/node-resolver-759xk" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308762 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d6a61ca-a15f-4742-8e81-6057c23e51c8-host-slash\") pod \"iptables-alerter-2cvn5\" (UID: \"0d6a61ca-a15f-4742-8e81-6057c23e51c8\") " pod="openshift-network-operator/iptables-alerter-2cvn5" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308791 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-run-ovn\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308815 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-var-lib-kubelet\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308855 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-sysctl-conf\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308894 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-default-certificate\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308921 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9446374-a7ea-4f95-baa4-d6dad6805b65-cnibin\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.308984 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk88t\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-kube-api-access-rk88t\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309009 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fdddab43-acaf-4153-bb52-3e60044a35dc-serving-cert\") pod \"console-operator-7cd56b8dbc-nbwj8\" (UID: \"fdddab43-acaf-4153-bb52-3e60044a35dc\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309029 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-stats-auth\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309051 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5djfp\" (UniqueName: \"kubernetes.io/projected/0d6a61ca-a15f-4742-8e81-6057c23e51c8-kube-api-access-5djfp\") pod \"iptables-alerter-2cvn5\" (UID: \"0d6a61ca-a15f-4742-8e81-6057c23e51c8\") " pod="openshift-network-operator/iptables-alerter-2cvn5" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309097 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-wtgzn\" (UID: \"66992144-51dd-4c1d-95c8-63862bca97cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309125 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6zpc\" (UniqueName: \"kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc\") pod \"network-check-target-d6rt6\" (UID: \"a5428d1a-8c8b-41ce-ad97-a729ec7a2e43\") " pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309149 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h7bv\" (UniqueName: \"kubernetes.io/projected/d4ca5142-5b8a-4406-9ed0-06f9edea89bc-kube-api-access-5h7bv\") pod \"ingress-canary-fnmsk\" (UID: \"d4ca5142-5b8a-4406-9ed0-06f9edea89bc\") " pod="openshift-ingress-canary/ingress-canary-fnmsk" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309167 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-default-certificate\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309178 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-log-socket\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309205 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fdddab43-acaf-4153-bb52-3e60044a35dc-serving-cert\") pod \"console-operator-7cd56b8dbc-nbwj8\" (UID: \"fdddab43-acaf-4153-bb52-3e60044a35dc\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309251 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-stats-auth\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309293 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-run-k8s-cni-cncf-io\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309391 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-var-lib-cni-multus\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309442 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309454 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-wtgzn\" (UID: \"66992144-51dd-4c1d-95c8-63862bca97cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309477 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs\") pod \"network-metrics-daemon-24p2b\" (UID: \"16a02457-151b-4e96-8118-e7695190b9be\") " pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309506 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvwvc\" (UniqueName: \"kubernetes.io/projected/7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d-kube-api-access-mvwvc\") pod \"node-resolver-759xk\" (UID: \"7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d\") " pod="openshift-dns/node-resolver-759xk" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309559 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2a12192e-f0c8-4354-b9fa-1b65b04ebc19-host\") pod \"node-ca-vsf9w\" (UID: \"2a12192e-f0c8-4354-b9fa-1b65b04ebc19\") " pod="openshift-image-registry/node-ca-vsf9w" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309606 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7r5b\" (UniqueName: \"kubernetes.io/projected/52ca5902-69f0-462f-ae35-cbbc7f378345-kube-api-access-n7r5b\") pod \"service-ca-operator-544d749c8c-fk7rp\" (UID: \"52ca5902-69f0-462f-ae35-cbbc7f378345\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309706 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-installation-pull-secrets\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309725 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wtgzn\" (UID: \"66992144-51dd-4c1d-95c8-63862bca97cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309747 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-cni-bin\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309788 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-kubernetes\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309806 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-systemd\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309822 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-var-lib-kubelet\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309854 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75kpm\" (UniqueName: \"kubernetes.io/projected/2f5f56fc-d2d4-4cef-86c0-0ac4b7a51e41-kube-api-access-75kpm\") pod \"kas-connection-checker-685895c7c9-nprbr\" (UID: \"2f5f56fc-d2d4-4cef-86c0-0ac4b7a51e41\") " pod="kube-system/kas-connection-checker-685895c7c9-nprbr" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309889 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-installation-pull-secrets\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309916 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9446374-a7ea-4f95-baa4-d6dad6805b65-system-cni-dir\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309960 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fb047779-ad10-4f7e-a45a-4a154ab60ad8-trusted-ca\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:42.309968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.309986 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/23d95081-1188-4002-997e-65ca57111fab-ovnkube-script-lib\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310024 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbwv9\" (UniqueName: \"kubernetes.io/projected/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-kube-api-access-sbwv9\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310054 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-sysconfig\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310095 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-service-ca-bundle\") pod \"periodic-gathering-vs88s-f66ls\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310192 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8ksd\" (UniqueName: \"kubernetes.io/projected/0e13113b-b4d2-42d7-9e1c-642a790bcd55-kube-api-access-j8ksd\") pod \"network-check-source-6f78d59f99-xsw6d\" (UID: \"0e13113b-b4d2-42d7-9e1c-642a790bcd55\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xsw6d" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310233 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/23d95081-1188-4002-997e-65ca57111fab-ovnkube-script-lib\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310246 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-service-ca-bundle\") pod \"periodic-gathering-vs88s-f66ls\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310270 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-tmp\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310303 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fdddab43-acaf-4153-bb52-3e60044a35dc-trusted-ca\") pod \"console-operator-7cd56b8dbc-nbwj8\" (UID: \"fdddab43-acaf-4153-bb52-3e60044a35dc\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310325 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-tmp\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310329 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-socket-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310369 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-run-systemd\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310387 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-tuned\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310405 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-bound-sa-token\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310447 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9446374-a7ea-4f95-baa4-d6dad6805b65-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310483 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/75bd9ac5-49ff-4c58-85c7-79b181150555-dbus\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310487 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-tuned\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310529 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-run-netns\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310561 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c59v8\" (UniqueName: \"kubernetes.io/projected/fdddab43-acaf-4153-bb52-3e60044a35dc-kube-api-access-c59v8\") pod \"console-operator-7cd56b8dbc-nbwj8\" (UID: \"fdddab43-acaf-4153-bb52-3e60044a35dc\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310588 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9446374-a7ea-4f95-baa4-d6dad6805b65-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310630 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-kubelet\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310663 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fb047779-ad10-4f7e-a45a-4a154ab60ad8-trusted-ca\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310662 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310680 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9446374-a7ea-4f95-baa4-d6dad6805b65-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310703 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5cd3297b-e3fe-40e0-be87-88a67236d9f0-crio-socket\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310733 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310762 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-trusted-ca\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310801 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dc5nm\" (UniqueName: \"kubernetes.io/projected/16a02457-151b-4e96-8118-e7695190b9be-kube-api-access-dc5nm\") pod \"network-metrics-daemon-24p2b\" (UID: \"16a02457-151b-4e96-8118-e7695190b9be\") " pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310801 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9446374-a7ea-4f95-baa4-d6dad6805b65-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310916 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4ca5142-5b8a-4406-9ed0-06f9edea89bc-cert\") pod \"ingress-canary-fnmsk\" (UID: \"d4ca5142-5b8a-4406-9ed0-06f9edea89bc\") " pod="openshift-ingress-canary/ingress-canary-fnmsk" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310934 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-socket-dir-parent\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.310969 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fdddab43-acaf-4153-bb52-3e60044a35dc-trusted-ca\") pod \"console-operator-7cd56b8dbc-nbwj8\" (UID: \"fdddab43-acaf-4153-bb52-3e60044a35dc\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.311351 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-trusted-ca\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.311346 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9446374-a7ea-4f95-baa4-d6dad6805b65-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.311425 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-device-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.311475 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4ca5142-5b8a-4406-9ed0-06f9edea89bc-cert\") pod \"ingress-canary-fnmsk\" (UID: \"d4ca5142-5b8a-4406-9ed0-06f9edea89bc\") " pod="openshift-ingress-canary/ingress-canary-fnmsk" Oct 30 12:51:42.311513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.311473 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-997qf\" (UniqueName: \"kubernetes.io/projected/0b1634e9-3028-4b6a-b651-9155018b3c5a-kube-api-access-997qf\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.311547 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8hrc\" (UniqueName: \"kubernetes.io/projected/23d95081-1188-4002-997e-65ca57111fab-kube-api-access-j8hrc\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.311722 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-tmp-dir\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.311796 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cd3297b-e3fe-40e0-be87-88a67236d9f0-data-volume\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.311827 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqfvq\" (UniqueName: \"kubernetes.io/projected/3e77ccc1-14a3-4f85-a214-854bd392733a-kube-api-access-pqfvq\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.311856 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9951ec02-ee4f-488c-8cab-3985569328f8-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr\" (UID: \"9951ec02-ee4f-488c-8cab-3985569328f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.311884 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-tmp-dir\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.311892 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5fjr\" (UniqueName: \"kubernetes.io/projected/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-kube-api-access-d5fjr\") pod \"periodic-gathering-vs88s-f66ls\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.311928 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdmq4\" (UniqueName: \"kubernetes.io/projected/66992144-51dd-4c1d-95c8-63862bca97cc-kube-api-access-mdmq4\") pod \"cluster-samples-operator-6ddb6c8b94-wtgzn\" (UID: \"66992144-51dd-4c1d-95c8-63862bca97cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.311965 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d-hosts-file\") pod \"node-resolver-759xk\" (UID: \"7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d\") " pod="openshift-dns/node-resolver-759xk" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.311992 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-sysctl-d\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.312023 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-host\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.312113 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d6a61ca-a15f-4742-8e81-6057c23e51c8-iptables-alerter-script\") pod \"iptables-alerter-2cvn5\" (UID: \"0d6a61ca-a15f-4742-8e81-6057c23e51c8\") " pod="openshift-network-operator/iptables-alerter-2cvn5" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.312156 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-run-ovn-kubernetes\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.312195 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/23d95081-1188-4002-997e-65ca57111fab-env-overrides\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.312207 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5cd3297b-e3fe-40e0-be87-88a67236d9f0-data-volume\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.312230 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-modprobe-d\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.312305 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fdddab43-acaf-4153-bb52-3e60044a35dc-config\") pod \"console-operator-7cd56b8dbc-nbwj8\" (UID: \"fdddab43-acaf-4153-bb52-3e60044a35dc\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.312625 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/23d95081-1188-4002-997e-65ca57111fab-env-overrides\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.312688 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-image-registry-private-configuration\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.312727 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-bound-sa-token\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.312764 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjl5n\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-kube-api-access-gjl5n\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.312797 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-serving-cert\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.312905 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d6a61ca-a15f-4742-8e81-6057c23e51c8-iptables-alerter-script\") pod \"iptables-alerter-2cvn5\" (UID: \"0d6a61ca-a15f-4742-8e81-6057c23e51c8\") " pod="openshift-network-operator/iptables-alerter-2cvn5" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.312910 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/109e0336-ecdb-4278-b5d3-72c7bdb80276-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-49h5k\" (UID: \"109e0336-ecdb-4278-b5d3-72c7bdb80276\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-49h5k" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.312950 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9446374-a7ea-4f95-baa4-d6dad6805b65-cni-binary-copy\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.312983 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/52ca5902-69f0-462f-ae35-cbbc7f378345-serving-cert\") pod \"service-ca-operator-544d749c8c-fk7rp\" (UID: \"52ca5902-69f0-462f-ae35-cbbc7f378345\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.313010 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-service-ca-bundle\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:42.313078 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.313046 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-slash\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.314278 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.313099 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-hostroot\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.314278 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.313119 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9951ec02-ee4f-488c-8cab-3985569328f8-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr\" (UID: \"9951ec02-ee4f-488c-8cab-3985569328f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr" Oct 30 12:51:42.314278 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.313201 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd65z\" (UniqueName: \"kubernetes.io/projected/e9446374-a7ea-4f95-baa4-d6dad6805b65-kube-api-access-wd65z\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:42.314278 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.313237 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htqvt\" (UniqueName: \"kubernetes.io/projected/9951ec02-ee4f-488c-8cab-3985569328f8-kube-api-access-htqvt\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr\" (UID: \"9951ec02-ee4f-488c-8cab-3985569328f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr" Oct 30 12:51:42.314278 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.313742 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9446374-a7ea-4f95-baa4-d6dad6805b65-cni-binary-copy\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:42.314278 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.313881 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-serving-cert\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:42.314278 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.313913 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/52ca5902-69f0-462f-ae35-cbbc7f378345-serving-cert\") pod \"service-ca-operator-544d749c8c-fk7rp\" (UID: \"52ca5902-69f0-462f-ae35-cbbc7f378345\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp" Oct 30 12:51:42.314278 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.314086 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-service-ca-bundle\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:42.314278 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.314058 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fdddab43-acaf-4153-bb52-3e60044a35dc-config\") pod \"console-operator-7cd56b8dbc-nbwj8\" (UID: \"fdddab43-acaf-4153-bb52-3e60044a35dc\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:42.314278 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.314259 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-image-registry-private-configuration\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:42.327819 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.327796 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4v4t\" (UniqueName: \"kubernetes.io/projected/1734d386-0cb1-4002-a954-b76b3cf62eaf-kube-api-access-j4v4t\") pod \"kas-connection-checker-685895c7c9-wsfvm\" (UID: \"1734d386-0cb1-4002-a954-b76b3cf62eaf\") " pod="kube-system/kas-connection-checker-685895c7c9-wsfvm" Oct 30 12:51:42.328030 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.328013 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmqmx\" (UniqueName: \"kubernetes.io/projected/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-kube-api-access-pmqmx\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.328474 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.328447 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg5w5\" (UniqueName: \"kubernetes.io/projected/9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462-kube-api-access-xg5w5\") pod \"insights-operator-7c4986d449-bbb6d\" (UID: \"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462\") " pod="openshift-insights/insights-operator-7c4986d449-bbb6d" Oct 30 12:51:42.328562 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.328531 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64858\" (UniqueName: \"kubernetes.io/projected/7d93802e-f47d-4319-b308-04fe84901551-kube-api-access-64858\") pod \"kas-connection-checker-685895c7c9-nzp7p\" (UID: \"7d93802e-f47d-4319-b308-04fe84901551\") " pod="kube-system/kas-connection-checker-685895c7c9-nzp7p" Oct 30 12:51:42.328608 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.328584 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhns5\" (UniqueName: \"kubernetes.io/projected/5cd3297b-e3fe-40e0-be87-88a67236d9f0-kube-api-access-fhns5\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:42.328832 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.328815 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scm4l\" (UniqueName: \"kubernetes.io/projected/f0d51a79-823b-4059-94c3-0535fc882a44-kube-api-access-scm4l\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:42.328863 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.328846 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mld9s\" (UniqueName: \"kubernetes.io/projected/2a12192e-f0c8-4354-b9fa-1b65b04ebc19-kube-api-access-mld9s\") pod \"node-ca-vsf9w\" (UID: \"2a12192e-f0c8-4354-b9fa-1b65b04ebc19\") " pod="openshift-image-registry/node-ca-vsf9w" Oct 30 12:51:42.341900 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.341878 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cgzb\" (UniqueName: \"kubernetes.io/projected/85617466-6b8e-45d4-a45e-e9650078c0be-kube-api-access-9cgzb\") pod \"migrator-66df5d496b-pjfzl\" (UID: \"85617466-6b8e-45d4-a45e-e9650078c0be\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-pjfzl" Oct 30 12:51:42.342772 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.342690 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vkl7\" (UniqueName: \"kubernetes.io/projected/3162c3d2-e86f-4ae1-8196-0414509fae9e-kube-api-access-7vkl7\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.342772 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.342709 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75kpm\" (UniqueName: \"kubernetes.io/projected/2f5f56fc-d2d4-4cef-86c0-0ac4b7a51e41-kube-api-access-75kpm\") pod \"kas-connection-checker-685895c7c9-nprbr\" (UID: \"2f5f56fc-d2d4-4cef-86c0-0ac4b7a51e41\") " pod="kube-system/kas-connection-checker-685895c7c9-nprbr" Oct 30 12:51:42.342772 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.342690 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbwv9\" (UniqueName: \"kubernetes.io/projected/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-kube-api-access-sbwv9\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:42.342772 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.342738 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8ksd\" (UniqueName: \"kubernetes.io/projected/0e13113b-b4d2-42d7-9e1c-642a790bcd55-kube-api-access-j8ksd\") pod \"network-check-source-6f78d59f99-xsw6d\" (UID: \"0e13113b-b4d2-42d7-9e1c-642a790bcd55\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-xsw6d" Oct 30 12:51:42.343006 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.342760 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w928x\" (UniqueName: \"kubernetes.io/projected/6d7c203f-f696-471f-baf8-32c0ad647baf-kube-api-access-w928x\") pod \"migrator-66df5d496b-q68tl\" (UID: \"6d7c203f-f696-471f-baf8-32c0ad647baf\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-q68tl" Oct 30 12:51:42.343006 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.342913 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6zpc\" (UniqueName: \"kubernetes.io/projected/a5428d1a-8c8b-41ce-ad97-a729ec7a2e43-kube-api-access-q6zpc\") pod \"network-check-target-d6rt6\" (UID: \"a5428d1a-8c8b-41ce-ad97-a729ec7a2e43\") " pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:42.344004 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.343226 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5djfp\" (UniqueName: \"kubernetes.io/projected/0d6a61ca-a15f-4742-8e81-6057c23e51c8-kube-api-access-5djfp\") pod \"iptables-alerter-2cvn5\" (UID: \"0d6a61ca-a15f-4742-8e81-6057c23e51c8\") " pod="openshift-network-operator/iptables-alerter-2cvn5" Oct 30 12:51:42.344004 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.343503 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srvwc\" (UniqueName: \"kubernetes.io/projected/1a8f7b67-6d17-4f5e-a79b-52481ea0fd48-kube-api-access-srvwc\") pod \"volume-data-source-validator-58c9f5f46d-bzh2m\" (UID: \"1a8f7b67-6d17-4f5e-a79b-52481ea0fd48\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-bzh2m" Oct 30 12:51:42.344004 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.343539 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7r5b\" (UniqueName: \"kubernetes.io/projected/52ca5902-69f0-462f-ae35-cbbc7f378345-kube-api-access-n7r5b\") pod \"service-ca-operator-544d749c8c-fk7rp\" (UID: \"52ca5902-69f0-462f-ae35-cbbc7f378345\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp" Oct 30 12:51:42.344004 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.343668 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5fjr\" (UniqueName: \"kubernetes.io/projected/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-kube-api-access-d5fjr\") pod \"periodic-gathering-vs88s-f66ls\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:51:42.344004 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.343697 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjl5n\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-kube-api-access-gjl5n\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:42.344004 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.343739 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqfvq\" (UniqueName: \"kubernetes.io/projected/3e77ccc1-14a3-4f85-a214-854bd392733a-kube-api-access-pqfvq\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:42.344004 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.343770 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvwvc\" (UniqueName: \"kubernetes.io/projected/7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d-kube-api-access-mvwvc\") pod \"node-resolver-759xk\" (UID: \"7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d\") " pod="openshift-dns/node-resolver-759xk" Oct 30 12:51:42.344004 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.343838 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk88t\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-kube-api-access-rk88t\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:42.344004 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.343857 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c59v8\" (UniqueName: \"kubernetes.io/projected/fdddab43-acaf-4153-bb52-3e60044a35dc-kube-api-access-c59v8\") pod \"console-operator-7cd56b8dbc-nbwj8\" (UID: \"fdddab43-acaf-4153-bb52-3e60044a35dc\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:42.344004 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.343943 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dc5nm\" (UniqueName: \"kubernetes.io/projected/16a02457-151b-4e96-8118-e7695190b9be-kube-api-access-dc5nm\") pod \"network-metrics-daemon-24p2b\" (UID: \"16a02457-151b-4e96-8118-e7695190b9be\") " pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:42.344004 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.343976 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htqvt\" (UniqueName: \"kubernetes.io/projected/9951ec02-ee4f-488c-8cab-3985569328f8-kube-api-access-htqvt\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr\" (UID: \"9951ec02-ee4f-488c-8cab-3985569328f8\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr" Oct 30 12:51:42.344380 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.344050 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdmq4\" (UniqueName: \"kubernetes.io/projected/66992144-51dd-4c1d-95c8-63862bca97cc-kube-api-access-mdmq4\") pod \"cluster-samples-operator-6ddb6c8b94-wtgzn\" (UID: \"66992144-51dd-4c1d-95c8-63862bca97cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:42.344380 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.344142 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h7bv\" (UniqueName: \"kubernetes.io/projected/d4ca5142-5b8a-4406-9ed0-06f9edea89bc-kube-api-access-5h7bv\") pod \"ingress-canary-fnmsk\" (UID: \"d4ca5142-5b8a-4406-9ed0-06f9edea89bc\") " pod="openshift-ingress-canary/ingress-canary-fnmsk" Oct 30 12:51:42.344380 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.344144 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8hrc\" (UniqueName: \"kubernetes.io/projected/23d95081-1188-4002-997e-65ca57111fab-kube-api-access-j8hrc\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.344380 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.344185 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-997qf\" (UniqueName: \"kubernetes.io/projected/0b1634e9-3028-4b6a-b651-9155018b3c5a-kube-api-access-997qf\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:42.351210 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.351192 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-bound-sa-token\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:42.352189 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.352169 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-bound-sa-token\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:42.352341 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.352324 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd65z\" (UniqueName: \"kubernetes.io/projected/e9446374-a7ea-4f95-baa4-d6dad6805b65-kube-api-access-wd65z\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:42.413874 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.413847 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d-hosts-file\") pod \"node-resolver-759xk\" (UID: \"7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d\") " pod="openshift-dns/node-resolver-759xk" Oct 30 12:51:42.414096 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.413884 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-sysctl-d\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.414096 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.413905 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-host\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.414096 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.413927 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-run-ovn-kubernetes\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.414096 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.413960 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-modprobe-d\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.414096 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.413980 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d-hosts-file\") pod \"node-resolver-759xk\" (UID: \"7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d\") " pod="openshift-dns/node-resolver-759xk" Oct 30 12:51:42.414096 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.413990 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-host\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.414096 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.413988 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/109e0336-ecdb-4278-b5d3-72c7bdb80276-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-49h5k\" (UID: \"109e0336-ecdb-4278-b5d3-72c7bdb80276\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-49h5k" Oct 30 12:51:42.414096 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.413994 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-run-ovn-kubernetes\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.414096 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414034 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-slash\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.414096 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414050 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-hostroot\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.414096 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414051 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-sysctl-d\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.414096 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414051 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-modprobe-d\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.414096 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414092 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-slash\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.414096 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414093 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414114 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-hostroot\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:42.414098 5029 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414131 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:42.414155 5029 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414160 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-registration-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414177 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-run-netns\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414179 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:42.414204 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/109e0336-ecdb-4278-b5d3-72c7bdb80276-networking-console-plugin-cert podName:109e0336-ecdb-4278-b5d3-72c7bdb80276 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:42.914188005 +0000 UTC m=+1.939829263 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/109e0336-ecdb-4278-b5d3-72c7bdb80276-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-49h5k" (UID: "109e0336-ecdb-4278-b5d3-72c7bdb80276") : secret "networking-console-plugin-cert" not found Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414220 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-system-cni-dir\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414230 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-registration-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414204 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-run-netns\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414245 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-cnibin\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414266 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-sys\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414279 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-system-cni-dir\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:42.414288 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls podName:f0d51a79-823b-4059-94c3-0535fc882a44 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:42.914271141 +0000 UTC m=+1.939912380 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6s9dp" (UID: "f0d51a79-823b-4059-94c3-0535fc882a44") : secret "cluster-monitoring-operator-tls" not found Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414309 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-sys\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414328 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-run-openvswitch\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414361 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-run-multus-certs\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414313 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-cnibin\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414386 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-metrics-tls\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414411 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-run-multus-certs\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414426 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e77ccc1-14a3-4f85-a214-854bd392733a-service-ca-bundle\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414444 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/75bd9ac5-49ff-4c58-85c7-79b181150555-kubelet-config\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414389 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-run-openvswitch\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414475 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-etc-openvswitch\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414492 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-cni-dir\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414493 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/75bd9ac5-49ff-4c58-85c7-79b181150555-kubelet-config\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414523 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-etc-openvswitch\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414531 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-cni-dir\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414548 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-conf-dir\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414570 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-lib-modules\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414585 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-var-lib-openvswitch\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414606 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-node-log\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414627 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-var-lib-cni-bin\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414633 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-var-lib-openvswitch\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414648 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-run\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414670 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-var-lib-cni-bin\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414670 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-node-log\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.414675 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414672 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414719 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-run\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414724 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-etc-selinux\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414684 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-conf-dir\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414729 5029 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414691 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-lib-modules\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414771 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414780 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-etc-selinux\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414808 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-cni-netd\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414842 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-cni-netd\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414893 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-etc-kubernetes\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414925 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-sys-fs\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414968 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-systemd-units\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414982 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-etc-kubernetes\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.414993 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5cd3297b-e3fe-40e0-be87-88a67236d9f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415027 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d6a61ca-a15f-4742-8e81-6057c23e51c8-host-slash\") pod \"iptables-alerter-2cvn5\" (UID: \"0d6a61ca-a15f-4742-8e81-6057c23e51c8\") " pod="openshift-network-operator/iptables-alerter-2cvn5" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415045 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e77ccc1-14a3-4f85-a214-854bd392733a-service-ca-bundle\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415087 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d6a61ca-a15f-4742-8e81-6057c23e51c8-host-slash\") pod \"iptables-alerter-2cvn5\" (UID: \"0d6a61ca-a15f-4742-8e81-6057c23e51c8\") " pod="openshift-network-operator/iptables-alerter-2cvn5" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415028 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-sys-fs\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415092 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-run-ovn\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415127 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-run-ovn\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415127 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-systemd-units\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415128 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-var-lib-kubelet\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415164 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-var-lib-kubelet\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415206 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-sysctl-conf\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415264 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9446374-a7ea-4f95-baa4-d6dad6805b65-cnibin\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415317 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9446374-a7ea-4f95-baa4-d6dad6805b65-cnibin\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415337 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-sysctl-conf\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415382 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-log-socket\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415340 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-log-socket\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415413 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5cd3297b-e3fe-40e0-be87-88a67236d9f0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415419 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-run-k8s-cni-cncf-io\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415455 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-var-lib-cni-multus\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415474 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-run-k8s-cni-cncf-io\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415480 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415505 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs\") pod \"network-metrics-daemon-24p2b\" (UID: \"16a02457-151b-4e96-8118-e7695190b9be\") " pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415512 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-var-lib-cni-multus\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415528 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2a12192e-f0c8-4354-b9fa-1b65b04ebc19-host\") pod \"node-ca-vsf9w\" (UID: \"2a12192e-f0c8-4354-b9fa-1b65b04ebc19\") " pod="openshift-image-registry/node-ca-vsf9w" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415572 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2a12192e-f0c8-4354-b9fa-1b65b04ebc19-host\") pod \"node-ca-vsf9w\" (UID: \"2a12192e-f0c8-4354-b9fa-1b65b04ebc19\") " pod="openshift-image-registry/node-ca-vsf9w" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:42.415590 5029 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:42.415639 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls podName:5cd3297b-e3fe-40e0-be87-88a67236d9f0 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:42.91562365 +0000 UTC m=+1.941264883 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l7z89" (UID: "5cd3297b-e3fe-40e0-be87-88a67236d9f0") : secret "insights-runtime-extractor-tls" not found Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415668 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wtgzn\" (UID: \"66992144-51dd-4c1d-95c8-63862bca97cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415703 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-cni-bin\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415730 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-kubernetes\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415754 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-systemd\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415769 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-cni-bin\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415776 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-var-lib-kubelet\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:42.415782 5029 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415814 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-kubernetes\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415822 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-var-lib-kubelet\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415841 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9446374-a7ea-4f95-baa4-d6dad6805b65-system-cni-dir\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:42.415851 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls podName:66992144-51dd-4c1d-95c8-63862bca97cc nodeName:}" failed. No retries permitted until 2026-10-30 12:51:42.915836213 +0000 UTC m=+1.941477442 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-wtgzn" (UID: "66992144-51dd-4c1d-95c8-63862bca97cc") : secret "samples-operator-tls" not found Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415852 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-systemd\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415886 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-sysconfig\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415883 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9446374-a7ea-4f95-baa4-d6dad6805b65-system-cni-dir\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415911 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-socket-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415929 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-run-systemd\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415947 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/75bd9ac5-49ff-4c58-85c7-79b181150555-dbus\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415947 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c4c9c543-f712-4a48-949f-d3f3bc8f77e1-etc-sysconfig\") pod \"tuned-qwj8g\" (UID: \"c4c9c543-f712-4a48-949f-d3f3bc8f77e1\") " pod="openshift-cluster-node-tuning-operator/tuned-qwj8g" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415975 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-run-netns\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.415987 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/75bd9ac5-49ff-4c58-85c7-79b181150555-dbus\") pod \"global-pull-secret-syncer-lf7b2\" (UID: \"75bd9ac5-49ff-4c58-85c7-79b181150555\") " pod="kube-system/global-pull-secret-syncer-lf7b2" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.416002 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-kubelet\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.416014 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-run-systemd\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.416019 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-socket-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.416026 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.416037 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-host-run-netns\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.416083 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-kubelet\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.416090 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5cd3297b-e3fe-40e0-be87-88a67236d9f0-crio-socket\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.416088 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/23d95081-1188-4002-997e-65ca57111fab-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cm78w\" (UID: \"23d95081-1188-4002-997e-65ca57111fab\") " pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.416116 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.416133 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-socket-dir-parent\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.416130 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5cd3297b-e3fe-40e0-be87-88a67236d9f0-crio-socket\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.416162 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9446374-a7ea-4f95-baa4-d6dad6805b65-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.416170 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3162c3d2-e86f-4ae1-8196-0414509fae9e-multus-socket-dir-parent\") pod \"multus-hjpbf\" (UID: \"3162c3d2-e86f-4ae1-8196-0414509fae9e\") " pod="openshift-multus/multus-hjpbf" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.416190 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-device-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:42.416229 5029 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.416266 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9446374-a7ea-4f95-baa4-d6dad6805b65-tuning-conf-dir\") pod \"multus-additional-cni-plugins-s2xpm\" (UID: \"e9446374-a7ea-4f95-baa4-d6dad6805b65\") " pod="openshift-multus/multus-additional-cni-plugins-s2xpm" Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:42.416271 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs podName:3e77ccc1-14a3-4f85-a214-854bd392733a nodeName:}" failed. No retries permitted until 2026-10-30 12:51:42.91625818 +0000 UTC m=+1.941899425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs") pod "router-default-767cffcd47-qx67t" (UID: "3e77ccc1-14a3-4f85-a214-854bd392733a") : secret "router-metrics-certs-default" not found Oct 30 12:51:42.416298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.416270 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0b1634e9-3028-4b6a-b651-9155018b3c5a-device-dir\") pod \"aws-ebs-csi-driver-node-hrfwr\" (UID: \"0b1634e9-3028-4b6a-b651-9155018b3c5a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hrfwr" Oct 30 12:51:42.418515 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.417465 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/38b6bfaf-2fb2-455b-9e7d-32586b3f216f-metrics-tls\") pod \"dns-default-cd6w9\" (UID: \"38b6bfaf-2fb2-455b-9e7d-32586b3f216f\") " pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:42.418515 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.417471 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls\") pod \"image-registry-5cd77977d7-wrk42\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:42.418515 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.417529 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls\") pod \"image-registry-5cfd489894-v5wlw\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:42.418515 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.417933 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/16a02457-151b-4e96-8118-e7695190b9be-metrics-certs\") pod \"network-metrics-daemon-24p2b\" (UID: \"16a02457-151b-4e96-8118-e7695190b9be\") " pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:42.466889 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.466868 5029 scope.go:122] "RemoveContainer" containerID="72e9b907fdc4c8a3603e98b9c0298a772b043c460a2a2a72a020710cd8e9eeeb" Oct 30 12:51:42.469806 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.469789 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:42.478448 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.478427 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-24p2b" Oct 30 12:51:42.479551 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.479533 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:42.479625 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.479617 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:42.761059 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.761033 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:42.821937 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.821909 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cfd489894-v5wlw"] Oct 30 12:51:42.827733 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:42.827706 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb047779_ad10_4f7e_a45a_4a154ab60ad8.slice/crio-b2743e8de853dd4dc9dcc324b6986ce79b5f01cc98431abdcbb266a19408463b WatchSource:0}: Error finding container b2743e8de853dd4dc9dcc324b6986ce79b5f01cc98431abdcbb266a19408463b: Status 404 returned error can't find the container with id b2743e8de853dd4dc9dcc324b6986ce79b5f01cc98431abdcbb266a19408463b Oct 30 12:51:42.833721 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.833698 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-24p2b"] Oct 30 12:51:42.842510 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:42.842485 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16a02457_151b_4e96_8118_e7695190b9be.slice/crio-2f0c06bcbaf43ecd968a611f3f9a421da207816335d7750f89541db0153f8e15 WatchSource:0}: Error finding container 2f0c06bcbaf43ecd968a611f3f9a421da207816335d7750f89541db0153f8e15: Status 404 returned error can't find the container with id 2f0c06bcbaf43ecd968a611f3f9a421da207816335d7750f89541db0153f8e15 Oct 30 12:51:42.844537 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.844518 5029 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 12:51:42.876659 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.876632 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cd77977d7-wrk42"] Oct 30 12:51:42.881016 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:42.880992 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5133b471_c5f8_4b35_b06c_1d8f429c3ba2.slice/crio-fcdab23e1e6361bcb45dfa5ecd318219bce98a3f25f5821342751dafb82ecfd0 WatchSource:0}: Error finding container fcdab23e1e6361bcb45dfa5ecd318219bce98a3f25f5821342751dafb82ecfd0: Status 404 returned error can't find the container with id fcdab23e1e6361bcb45dfa5ecd318219bce98a3f25f5821342751dafb82ecfd0 Oct 30 12:51:42.903184 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.903161 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cd6w9"] Oct 30 12:51:42.908193 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:42.908172 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38b6bfaf_2fb2_455b_9e7d_32586b3f216f.slice/crio-e51bf7bc2e08991af0210353235bd6fa5787f7850a611632218b0adabaee3a7a WatchSource:0}: Error finding container e51bf7bc2e08991af0210353235bd6fa5787f7850a611632218b0adabaee3a7a: Status 404 returned error can't find the container with id e51bf7bc2e08991af0210353235bd6fa5787f7850a611632218b0adabaee3a7a Oct 30 12:51:42.921048 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.921019 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:42.921202 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.921092 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wtgzn\" (UID: \"66992144-51dd-4c1d-95c8-63862bca97cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:42.921202 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.921141 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:42.921202 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:42.921171 5029 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 12:51:42.921202 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.921191 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/109e0336-ecdb-4278-b5d3-72c7bdb80276-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-49h5k\" (UID: \"109e0336-ecdb-4278-b5d3-72c7bdb80276\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-49h5k" Oct 30 12:51:42.921627 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:42.921235 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls podName:5cd3297b-e3fe-40e0-be87-88a67236d9f0 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:43.921216065 +0000 UTC m=+2.946857310 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l7z89" (UID: "5cd3297b-e3fe-40e0-be87-88a67236d9f0") : secret "insights-runtime-extractor-tls" not found Oct 30 12:51:42.921627 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:42.921259 5029 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 12:51:42.921627 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:42.921318 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls podName:66992144-51dd-4c1d-95c8-63862bca97cc nodeName:}" failed. No retries permitted until 2026-10-30 12:51:43.921301533 +0000 UTC m=+2.946942775 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-wtgzn" (UID: "66992144-51dd-4c1d-95c8-63862bca97cc") : secret "samples-operator-tls" not found Oct 30 12:51:42.921627 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.921261 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:42.921627 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:42.921325 5029 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 12:51:42.921627 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:42.921341 5029 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 12:51:42.921627 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:42.921365 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls podName:f0d51a79-823b-4059-94c3-0535fc882a44 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:43.921353127 +0000 UTC m=+2.946994400 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6s9dp" (UID: "f0d51a79-823b-4059-94c3-0535fc882a44") : secret "cluster-monitoring-operator-tls" not found Oct 30 12:51:42.921627 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:42.921390 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs podName:3e77ccc1-14a3-4f85-a214-854bd392733a nodeName:}" failed. No retries permitted until 2026-10-30 12:51:43.921378611 +0000 UTC m=+2.947019849 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs") pod "router-default-767cffcd47-qx67t" (UID: "3e77ccc1-14a3-4f85-a214-854bd392733a") : secret "router-metrics-certs-default" not found Oct 30 12:51:42.923970 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:42.923945 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/109e0336-ecdb-4278-b5d3-72c7bdb80276-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-49h5k\" (UID: \"109e0336-ecdb-4278-b5d3-72c7bdb80276\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-49h5k" Oct 30 12:51:43.073273 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:43.073197 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-49h5k" Oct 30 12:51:43.147422 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:43.147389 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" event={"ID":"5133b471-c5f8-4b35-b06c-1d8f429c3ba2","Type":"ContainerStarted","Data":"b1bf434d8fcc43c9e083309fd752d13649e1b2c42efb54bba6142e0f5b723131"} Oct 30 12:51:43.147529 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:43.147455 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" event={"ID":"5133b471-c5f8-4b35-b06c-1d8f429c3ba2","Type":"ContainerStarted","Data":"fcdab23e1e6361bcb45dfa5ecd318219bce98a3f25f5821342751dafb82ecfd0"} Oct 30 12:51:43.147529 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:43.147514 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:51:43.149227 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:43.149197 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" event={"ID":"fb047779-ad10-4f7e-a45a-4a154ab60ad8","Type":"ContainerStarted","Data":"d60b37d749dd691d757bf67acf9e3902e10daf5362e1164741d9df169c2cf24e"} Oct 30 12:51:43.149324 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:43.149250 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" event={"ID":"fb047779-ad10-4f7e-a45a-4a154ab60ad8","Type":"ContainerStarted","Data":"b2743e8de853dd4dc9dcc324b6986ce79b5f01cc98431abdcbb266a19408463b"} Oct 30 12:51:43.151697 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:43.151677 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/2.log" Oct 30 12:51:43.152474 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:43.152457 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/1.log" Oct 30 12:51:43.152564 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:43.152488 5029 generic.go:356] "Generic (PLEG): container finished" podID="fdddab43-acaf-4153-bb52-3e60044a35dc" containerID="93514973f6890f9cb8cdfe6dbd6f96b5e9484224ea4128bd466471b098cc6dc5" exitCode=255 Oct 30 12:51:43.152619 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:43.152605 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" event={"ID":"fdddab43-acaf-4153-bb52-3e60044a35dc","Type":"ContainerDied","Data":"93514973f6890f9cb8cdfe6dbd6f96b5e9484224ea4128bd466471b098cc6dc5"} Oct 30 12:51:43.152683 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:43.152647 5029 scope.go:122] "RemoveContainer" containerID="72e9b907fdc4c8a3603e98b9c0298a772b043c460a2a2a72a020710cd8e9eeeb" Oct 30 12:51:43.152991 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:43.152960 5029 scope.go:122] "RemoveContainer" containerID="93514973f6890f9cb8cdfe6dbd6f96b5e9484224ea4128bd466471b098cc6dc5" Oct 30 12:51:43.153204 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:43.153168 5029 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-nbwj8_openshift-console-operator(fdddab43-acaf-4153-bb52-3e60044a35dc)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" podUID="fdddab43-acaf-4153-bb52-3e60044a35dc" Oct 30 12:51:43.155805 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:43.155784 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cd6w9" event={"ID":"38b6bfaf-2fb2-455b-9e7d-32586b3f216f","Type":"ContainerStarted","Data":"e51bf7bc2e08991af0210353235bd6fa5787f7850a611632218b0adabaee3a7a"} Oct 30 12:51:43.157058 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:43.157035 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-24p2b" event={"ID":"16a02457-151b-4e96-8118-e7695190b9be","Type":"ContainerStarted","Data":"2f0c06bcbaf43ecd968a611f3f9a421da207816335d7750f89541db0153f8e15"} Oct 30 12:51:43.378749 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:43.378676 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-49h5k"] Oct 30 12:51:43.385419 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:43.385388 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod109e0336_ecdb_4278_b5d3_72c7bdb80276.slice/crio-894d0c5fc9e31641045df8dd4a791acebacb0c5f7c9511c82f63956d0404be88 WatchSource:0}: Error finding container 894d0c5fc9e31641045df8dd4a791acebacb0c5f7c9511c82f63956d0404be88: Status 404 returned error can't find the container with id 894d0c5fc9e31641045df8dd4a791acebacb0c5f7c9511c82f63956d0404be88 Oct 30 12:51:43.933111 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:43.933055 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:43.933231 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:43.933157 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:43.933294 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:43.933244 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:43.933294 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:43.933277 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wtgzn\" (UID: \"66992144-51dd-4c1d-95c8-63862bca97cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:43.933432 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:43.933413 5029 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 12:51:43.933493 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:43.933482 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls podName:66992144-51dd-4c1d-95c8-63862bca97cc nodeName:}" failed. No retries permitted until 2026-10-30 12:51:45.933463091 +0000 UTC m=+4.959104333 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-wtgzn" (UID: "66992144-51dd-4c1d-95c8-63862bca97cc") : secret "samples-operator-tls" not found Oct 30 12:51:43.933571 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:43.933558 5029 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 12:51:43.933620 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:43.933602 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs podName:3e77ccc1-14a3-4f85-a214-854bd392733a nodeName:}" failed. No retries permitted until 2026-10-30 12:51:45.933590394 +0000 UTC m=+4.959231623 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs") pod "router-default-767cffcd47-qx67t" (UID: "3e77ccc1-14a3-4f85-a214-854bd392733a") : secret "router-metrics-certs-default" not found Oct 30 12:51:43.933682 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:43.933656 5029 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 12:51:43.933735 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:43.933685 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls podName:f0d51a79-823b-4059-94c3-0535fc882a44 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:45.933676144 +0000 UTC m=+4.959317372 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6s9dp" (UID: "f0d51a79-823b-4059-94c3-0535fc882a44") : secret "cluster-monitoring-operator-tls" not found Oct 30 12:51:43.933789 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:43.933739 5029 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 12:51:43.933789 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:43.933773 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls podName:5cd3297b-e3fe-40e0-be87-88a67236d9f0 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:45.9337606 +0000 UTC m=+4.959401832 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l7z89" (UID: "5cd3297b-e3fe-40e0-be87-88a67236d9f0") : secret "insights-runtime-extractor-tls" not found Oct 30 12:51:44.164696 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:44.164659 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-24p2b" event={"ID":"16a02457-151b-4e96-8118-e7695190b9be","Type":"ContainerStarted","Data":"160ef25e721fb4169c0dfc5f8098f24e4478639b2125b07cc145c4a704e1cc1b"} Oct 30 12:51:44.165168 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:44.164764 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-24p2b" event={"ID":"16a02457-151b-4e96-8118-e7695190b9be","Type":"ContainerStarted","Data":"bda39afebbc1164e7e7c68905157f30424ce18bc2eed572ef59e61ce3fdd3f2a"} Oct 30 12:51:44.166523 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:44.166494 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-49h5k" event={"ID":"109e0336-ecdb-4278-b5d3-72c7bdb80276","Type":"ContainerStarted","Data":"894d0c5fc9e31641045df8dd4a791acebacb0c5f7c9511c82f63956d0404be88"} Oct 30 12:51:44.168492 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:44.168470 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/2.log" Oct 30 12:51:44.168615 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:44.168598 5029 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 12:51:45.174155 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:45.174108 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cd6w9" event={"ID":"38b6bfaf-2fb2-455b-9e7d-32586b3f216f","Type":"ContainerStarted","Data":"3d895aefdbab371b4a055234ec613e32949ba9e5b0397b3f4b50abbcc3cbd0cd"} Oct 30 12:51:45.174560 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:45.174171 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cd6w9" event={"ID":"38b6bfaf-2fb2-455b-9e7d-32586b3f216f","Type":"ContainerStarted","Data":"cca29dcf7316ef1faaba7d3a3ebd48a1cc4f5f92d5dfda6b997f4c1b0ec8a91e"} Oct 30 12:51:45.174560 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:45.174203 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:45.175628 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:45.175601 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-49h5k" event={"ID":"109e0336-ecdb-4278-b5d3-72c7bdb80276","Type":"ContainerStarted","Data":"91df9b33eabc443f69e233bdac95110c793d01293ceb844104d8538b409a2974"} Oct 30 12:51:45.952804 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:45.952766 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:45.952988 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:45.952826 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wtgzn\" (UID: \"66992144-51dd-4c1d-95c8-63862bca97cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:45.952988 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:45.952879 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:45.952988 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:45.952934 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:45.953165 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:45.953133 5029 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 12:51:45.953242 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:45.953215 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls podName:f0d51a79-823b-4059-94c3-0535fc882a44 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:49.953195744 +0000 UTC m=+8.978836971 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-6s9dp" (UID: "f0d51a79-823b-4059-94c3-0535fc882a44") : secret "cluster-monitoring-operator-tls" not found Oct 30 12:51:45.953660 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:45.953633 5029 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 12:51:45.953777 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:45.953695 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls podName:5cd3297b-e3fe-40e0-be87-88a67236d9f0 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:49.953680358 +0000 UTC m=+8.979321590 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l7z89" (UID: "5cd3297b-e3fe-40e0-be87-88a67236d9f0") : secret "insights-runtime-extractor-tls" not found Oct 30 12:51:45.954558 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:45.954532 5029 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 12:51:45.954670 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:45.954616 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs podName:3e77ccc1-14a3-4f85-a214-854bd392733a nodeName:}" failed. No retries permitted until 2026-10-30 12:51:49.954599702 +0000 UTC m=+8.980240948 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs") pod "router-default-767cffcd47-qx67t" (UID: "3e77ccc1-14a3-4f85-a214-854bd392733a") : secret "router-metrics-certs-default" not found Oct 30 12:51:45.957162 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:45.957135 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/66992144-51dd-4c1d-95c8-63862bca97cc-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wtgzn\" (UID: \"66992144-51dd-4c1d-95c8-63862bca97cc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:46.067949 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:46.067914 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" Oct 30 12:51:46.264264 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:46.264195 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:46.264619 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:46.264381 5029 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 12:51:46.390226 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:46.390195 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn"] Oct 30 12:51:47.184583 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:47.184551 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" event={"ID":"66992144-51dd-4c1d-95c8-63862bca97cc","Type":"ContainerStarted","Data":"18dee045eee8168ba439ecc1effcb18eaaef95535d2ab1869c9620267c3276ea"} Oct 30 12:51:48.189623 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:48.189552 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" event={"ID":"66992144-51dd-4c1d-95c8-63862bca97cc","Type":"ContainerStarted","Data":"0233e2bc7427f52e6576d833bf022676877fdc9da0a5e5f7d4b151917d4bbac4"} Oct 30 12:51:48.189623 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:48.189613 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wtgzn" event={"ID":"66992144-51dd-4c1d-95c8-63862bca97cc","Type":"ContainerStarted","Data":"68a7026a3cef1f47220bd57cd379222dc3ef324d83594f36c0698965b827261d"} Oct 30 12:51:49.322873 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:49.322842 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:51:49.872852 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:49.872808 5029 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:49.873226 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:49.873212 5029 scope.go:122] "RemoveContainer" containerID="93514973f6890f9cb8cdfe6dbd6f96b5e9484224ea4128bd466471b098cc6dc5" Oct 30 12:51:49.873433 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:49.873415 5029 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-nbwj8_openshift-console-operator(fdddab43-acaf-4153-bb52-3e60044a35dc)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" podUID="fdddab43-acaf-4153-bb52-3e60044a35dc" Oct 30 12:51:49.988460 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:49.988429 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:49.988652 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:49.988485 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:49.988652 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:49.988566 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:49.988772 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:49.988674 5029 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 12:51:49.988772 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:49.988741 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls podName:5cd3297b-e3fe-40e0-be87-88a67236d9f0 nodeName:}" failed. No retries permitted until 2026-10-30 12:51:57.988725479 +0000 UTC m=+17.014366707 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l7z89" (UID: "5cd3297b-e3fe-40e0-be87-88a67236d9f0") : secret "insights-runtime-extractor-tls" not found Oct 30 12:51:49.990836 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:49.990808 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f0d51a79-823b-4059-94c3-0535fc882a44-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-6s9dp\" (UID: \"f0d51a79-823b-4059-94c3-0535fc882a44\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:49.990836 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:49.990833 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3e77ccc1-14a3-4f85-a214-854bd392733a-metrics-certs\") pod \"router-default-767cffcd47-qx67t\" (UID: \"3e77ccc1-14a3-4f85-a214-854bd392733a\") " pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:50.149941 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:50.149856 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cm78w" Oct 30 12:51:50.266305 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:50.266280 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:50.268889 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:50.268873 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" Oct 30 12:51:50.566534 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:50.566501 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp"] Oct 30 12:51:50.571531 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:50.571492 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0d51a79_823b_4059_94c3_0535fc882a44.slice/crio-aa94276398f4e8afef9349505e4b86378df4c33869d6bbcd188e4e06349efb00 WatchSource:0}: Error finding container aa94276398f4e8afef9349505e4b86378df4c33869d6bbcd188e4e06349efb00: Status 404 returned error can't find the container with id aa94276398f4e8afef9349505e4b86378df4c33869d6bbcd188e4e06349efb00 Oct 30 12:51:50.584287 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:50.584265 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-767cffcd47-qx67t"] Oct 30 12:51:50.589833 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:50.589806 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e77ccc1_14a3_4f85_a214_854bd392733a.slice/crio-f57cd186f42c8dcd6f49953d7bf924a0e5c8de4651d843f1937ceea5b6a81ea4 WatchSource:0}: Error finding container f57cd186f42c8dcd6f49953d7bf924a0e5c8de4651d843f1937ceea5b6a81ea4: Status 404 returned error can't find the container with id f57cd186f42c8dcd6f49953d7bf924a0e5c8de4651d843f1937ceea5b6a81ea4 Oct 30 12:51:51.166992 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:51.166962 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:51.167440 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:51.167423 5029 scope.go:122] "RemoveContainer" containerID="93514973f6890f9cb8cdfe6dbd6f96b5e9484224ea4128bd466471b098cc6dc5" Oct 30 12:51:51.167657 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:51:51.167635 5029 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-nbwj8_openshift-console-operator(fdddab43-acaf-4153-bb52-3e60044a35dc)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" podUID="fdddab43-acaf-4153-bb52-3e60044a35dc" Oct 30 12:51:51.202456 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:51.202420 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-767cffcd47-qx67t" event={"ID":"3e77ccc1-14a3-4f85-a214-854bd392733a","Type":"ContainerStarted","Data":"6936cf5901ff7dadff5a0aaefc4341b529ea7bf49c3d0725de099b8ed1f017af"} Oct 30 12:51:51.202588 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:51.202478 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-767cffcd47-qx67t" event={"ID":"3e77ccc1-14a3-4f85-a214-854bd392733a","Type":"ContainerStarted","Data":"f57cd186f42c8dcd6f49953d7bf924a0e5c8de4651d843f1937ceea5b6a81ea4"} Oct 30 12:51:51.203903 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:51.203878 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" event={"ID":"f0d51a79-823b-4059-94c3-0535fc882a44","Type":"ContainerStarted","Data":"aa94276398f4e8afef9349505e4b86378df4c33869d6bbcd188e4e06349efb00"} Oct 30 12:51:51.267345 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:51.267312 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:51.270396 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:51.270375 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:52.208490 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:52.208453 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-6s9dp" event={"ID":"f0d51a79-823b-4059-94c3-0535fc882a44","Type":"ContainerStarted","Data":"10f61c8ca895732483f4a223142414fd513b914be8030ef61c218ad2b2cc8962"} Oct 30 12:51:52.298400 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:52.298313 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:51:53.212899 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:53.212875 5029 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 12:51:54.082661 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:54.082633 5029 scope.go:122] "RemoveContainer" containerID="93514973f6890f9cb8cdfe6dbd6f96b5e9484224ea4128bd466471b098cc6dc5" Oct 30 12:51:54.218911 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:54.218883 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/2.log" Oct 30 12:51:54.219288 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:54.218941 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" event={"ID":"fdddab43-acaf-4153-bb52-3e60044a35dc","Type":"ContainerStarted","Data":"c35bcfe7e273b59f3a1dd50d2ec86bf657f671268b7853903064183035522a89"} Oct 30 12:51:54.219288 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:54.219260 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:54.220230 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:54.220204 5029 patch_prober.go:29] interesting pod/console-operator-7cd56b8dbc-nbwj8 container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.9:8443/readyz\": dial tcp 10.132.0.9:8443: connect: connection refused" start-of-body= Oct 30 12:51:54.220327 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:54.220255 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" podUID="fdddab43-acaf-4153-bb52-3e60044a35dc" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.9:8443/readyz\": dial tcp 10.132.0.9:8443: connect: connection refused" Oct 30 12:51:55.182157 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:55.182128 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cd6w9" Oct 30 12:51:55.227454 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:55.227421 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-nbwj8" Oct 30 12:51:56.191865 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:56.191829 5029 patch_prober.go:29] interesting pod/image-registry-5cd77977d7-wrk42 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Oct 30 12:51:56.192015 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:56.191889 5029 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" podUID="5133b471-c5f8-4b35-b06c-1d8f429c3ba2" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 12:51:56.253784 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:56.253751 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5cfd489894-v5wlw"] Oct 30 12:51:56.690135 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:56.690103 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:56.690281 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:56.690250 5029 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 12:51:56.691429 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:56.691408 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-767cffcd47-qx67t" Oct 30 12:51:58.053612 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:58.053577 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:58.055849 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:58.055822 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5cd3297b-e3fe-40e0-be87-88a67236d9f0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l7z89\" (UID: \"5cd3297b-e3fe-40e0-be87-88a67236d9f0\") " pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:58.081970 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:58.081940 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-l7z89" Oct 30 12:51:58.382319 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:58.382288 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-l7z89"] Oct 30 12:51:58.387393 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:51:58.387369 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cd3297b_e3fe_40e0_be87_88a67236d9f0.slice/crio-d41fd33563700849885e8c44d8bd653f3c13202bbddce545bf904cf198933214 WatchSource:0}: Error finding container d41fd33563700849885e8c44d8bd653f3c13202bbddce545bf904cf198933214: Status 404 returned error can't find the container with id d41fd33563700849885e8c44d8bd653f3c13202bbddce545bf904cf198933214 Oct 30 12:51:59.241653 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:59.241567 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l7z89" event={"ID":"5cd3297b-e3fe-40e0-be87-88a67236d9f0","Type":"ContainerStarted","Data":"7acf9797ba5f873b52107b58ac45b7a38c4bccc3d30d7a6e387a8bf1d18b00bb"} Oct 30 12:51:59.241653 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:59.241633 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l7z89" event={"ID":"5cd3297b-e3fe-40e0-be87-88a67236d9f0","Type":"ContainerStarted","Data":"8d793875209d24b66d756dcd1af48b3d43345acdae9c233381828a3899c9193c"} Oct 30 12:51:59.242056 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:51:59.241662 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l7z89" event={"ID":"5cd3297b-e3fe-40e0-be87-88a67236d9f0","Type":"ContainerStarted","Data":"d41fd33563700849885e8c44d8bd653f3c13202bbddce545bf904cf198933214"} Oct 30 12:52:00.249347 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:00.249310 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l7z89" event={"ID":"5cd3297b-e3fe-40e0-be87-88a67236d9f0","Type":"ContainerStarted","Data":"fe9afe7d7aef2e2e4631dd243c5e67a8dfbe0a51d72b2085b6b64c21cff86ef4"} Oct 30 12:52:04.172623 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.172594 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:52:04.751913 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.751880 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zwmzb"] Oct 30 12:52:04.776245 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.776221 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.804807 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.804781 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5a0202a8-58b5-4cb3-82b4-697263f11ad9-node-exporter-accelerators-collector-config\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.804955 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.804815 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5a0202a8-58b5-4cb3-82b4-697263f11ad9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.804955 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.804865 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5a0202a8-58b5-4cb3-82b4-697263f11ad9-node-exporter-textfile\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.805100 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.804993 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5a0202a8-58b5-4cb3-82b4-697263f11ad9-root\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.805100 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.805032 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a0202a8-58b5-4cb3-82b4-697263f11ad9-sys\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.805100 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.805056 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a0202a8-58b5-4cb3-82b4-697263f11ad9-metrics-client-ca\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.805243 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.805124 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qkdh\" (UniqueName: \"kubernetes.io/projected/5a0202a8-58b5-4cb3-82b4-697263f11ad9-kube-api-access-7qkdh\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.805243 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.805201 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5a0202a8-58b5-4cb3-82b4-697263f11ad9-node-exporter-wtmp\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.805243 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.805231 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5a0202a8-58b5-4cb3-82b4-697263f11ad9-node-exporter-tls\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.826536 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.826503 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 30 12:52:04.826536 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.826510 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 30 12:52:04.870419 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.870386 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 30 12:52:04.870548 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.870443 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 30 12:52:04.870548 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.870533 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rwc4t\"" Oct 30 12:52:04.905819 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.905788 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5a0202a8-58b5-4cb3-82b4-697263f11ad9-node-exporter-accelerators-collector-config\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.905819 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.905820 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5a0202a8-58b5-4cb3-82b4-697263f11ad9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.906033 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.905851 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5a0202a8-58b5-4cb3-82b4-697263f11ad9-node-exporter-textfile\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.906033 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.905880 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5a0202a8-58b5-4cb3-82b4-697263f11ad9-root\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.906033 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.905907 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a0202a8-58b5-4cb3-82b4-697263f11ad9-sys\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.906033 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.905929 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a0202a8-58b5-4cb3-82b4-697263f11ad9-metrics-client-ca\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.906033 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.905963 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qkdh\" (UniqueName: \"kubernetes.io/projected/5a0202a8-58b5-4cb3-82b4-697263f11ad9-kube-api-access-7qkdh\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.906033 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.906015 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5a0202a8-58b5-4cb3-82b4-697263f11ad9-node-exporter-wtmp\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.906033 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.906017 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a0202a8-58b5-4cb3-82b4-697263f11ad9-sys\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.906424 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.906045 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5a0202a8-58b5-4cb3-82b4-697263f11ad9-node-exporter-tls\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.906424 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.906028 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5a0202a8-58b5-4cb3-82b4-697263f11ad9-root\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.906424 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:52:04.906145 5029 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Oct 30 12:52:04.906424 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:52:04.906208 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a0202a8-58b5-4cb3-82b4-697263f11ad9-node-exporter-tls podName:5a0202a8-58b5-4cb3-82b4-697263f11ad9 nodeName:}" failed. No retries permitted until 2026-10-30 12:52:05.406189142 +0000 UTC m=+24.431830386 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/5a0202a8-58b5-4cb3-82b4-697263f11ad9-node-exporter-tls") pod "node-exporter-zwmzb" (UID: "5a0202a8-58b5-4cb3-82b4-697263f11ad9") : secret "node-exporter-tls" not found Oct 30 12:52:04.906424 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.906242 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5a0202a8-58b5-4cb3-82b4-697263f11ad9-node-exporter-wtmp\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.906424 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.906301 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5a0202a8-58b5-4cb3-82b4-697263f11ad9-node-exporter-textfile\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.906715 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.906509 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5a0202a8-58b5-4cb3-82b4-697263f11ad9-node-exporter-accelerators-collector-config\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.906715 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.906590 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a0202a8-58b5-4cb3-82b4-697263f11ad9-metrics-client-ca\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.908233 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.908213 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5a0202a8-58b5-4cb3-82b4-697263f11ad9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:04.935504 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:04.935481 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qkdh\" (UniqueName: \"kubernetes.io/projected/5a0202a8-58b5-4cb3-82b4-697263f11ad9-kube-api-access-7qkdh\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:05.411142 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:05.411108 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5a0202a8-58b5-4cb3-82b4-697263f11ad9-node-exporter-tls\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:05.413502 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:05.413476 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5a0202a8-58b5-4cb3-82b4-697263f11ad9-node-exporter-tls\") pod \"node-exporter-zwmzb\" (UID: \"5a0202a8-58b5-4cb3-82b4-697263f11ad9\") " pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:05.452383 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:05.452356 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zwmzb" Oct 30 12:52:05.491994 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:52:05.491968 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a0202a8_58b5_4cb3_82b4_697263f11ad9.slice/crio-44bd0e74e4bb84461988456547a6511b9beafaee2bc065dc40be824977f656c1 WatchSource:0}: Error finding container 44bd0e74e4bb84461988456547a6511b9beafaee2bc065dc40be824977f656c1: Status 404 returned error can't find the container with id 44bd0e74e4bb84461988456547a6511b9beafaee2bc065dc40be824977f656c1 Oct 30 12:52:05.894128 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:05.894102 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vjn55" Oct 30 12:52:06.260949 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:06.260865 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:52:06.274783 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:06.274738 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zwmzb" event={"ID":"5a0202a8-58b5-4cb3-82b4-697263f11ad9","Type":"ContainerStarted","Data":"44bd0e74e4bb84461988456547a6511b9beafaee2bc065dc40be824977f656c1"} Oct 30 12:52:07.280116 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:07.280054 5029 generic.go:356] "Generic (PLEG): container finished" podID="5a0202a8-58b5-4cb3-82b4-697263f11ad9" containerID="14c3549ab8285c8c16f90bfc0ec7ab8e5ee5213a79bf1e5992072755b6f55425" exitCode=0 Oct 30 12:52:07.280517 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:07.280136 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zwmzb" event={"ID":"5a0202a8-58b5-4cb3-82b4-697263f11ad9","Type":"ContainerDied","Data":"14c3549ab8285c8c16f90bfc0ec7ab8e5ee5213a79bf1e5992072755b6f55425"} Oct 30 12:52:08.286981 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:08.286949 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zwmzb" event={"ID":"5a0202a8-58b5-4cb3-82b4-697263f11ad9","Type":"ContainerStarted","Data":"07ba3de47f717cd730fd925cd0e466f443f0100888e24b17de5104b5094f3a66"} Oct 30 12:52:08.287400 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:08.287024 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zwmzb" event={"ID":"5a0202a8-58b5-4cb3-82b4-697263f11ad9","Type":"ContainerStarted","Data":"3e824f7c459d4820e5f60bdb177b8e9016d3846d4b05f0751e492a54f0016546"} Oct 30 12:52:08.362226 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:08.362175 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zwmzb" podStartSLOduration=3.025473007 podStartE2EDuration="4.362160625s" totalImagesPullingTime="1.250192665s" totalInitContainerRuntime="86.494953ms" isStatefulPod=true podCreationTimestamp="2026-10-30 12:52:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:52:08.361330786 +0000 UTC m=+27.386972035" watchObservedRunningTime="2026-10-30 12:52:08.362160625 +0000 UTC m=+27.387801935" Oct 30 12:52:09.325455 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:09.325427 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-d6rt6" Oct 30 12:52:14.877803 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:52:14.877765 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 12:52:18.562381 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:18.562345 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5cd77977d7-wrk42"] Oct 30 12:52:21.275087 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.275032 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" podUID="fb047779-ad10-4f7e-a45a-4a154ab60ad8" containerName="registry" containerID="cri-o://d60b37d749dd691d757bf67acf9e3902e10daf5362e1164741d9df169c2cf24e" gracePeriod=30 Oct 30 12:52:21.530248 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.530191 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:52:21.652256 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.652215 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fb047779-ad10-4f7e-a45a-4a154ab60ad8-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fb047779-ad10-4f7e-a45a-4a154ab60ad8-installation-pull-secrets\") pod \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " Oct 30 12:52:21.652256 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.652268 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-certificates\") pod \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " Oct 30 12:52:21.652499 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.652290 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-kube-api-access-rk88t\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-kube-api-access-rk88t\") pod \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " Oct 30 12:52:21.652499 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.652322 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-bound-sa-token\") pod \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " Oct 30 12:52:21.652499 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.652385 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fb047779-ad10-4f7e-a45a-4a154ab60ad8-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fb047779-ad10-4f7e-a45a-4a154ab60ad8-image-registry-private-configuration\") pod \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " Oct 30 12:52:21.652499 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.652420 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb047779-ad10-4f7e-a45a-4a154ab60ad8-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fb047779-ad10-4f7e-a45a-4a154ab60ad8-trusted-ca\") pod \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " Oct 30 12:52:21.652499 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.652490 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb047779-ad10-4f7e-a45a-4a154ab60ad8-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fb047779-ad10-4f7e-a45a-4a154ab60ad8-ca-trust-extracted\") pod \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " Oct 30 12:52:21.652780 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.652536 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls\") pod \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\" (UID: \"fb047779-ad10-4f7e-a45a-4a154ab60ad8\") " Oct 30 12:52:21.652780 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.652690 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-certificates" pod "fb047779-ad10-4f7e-a45a-4a154ab60ad8" (UID: "fb047779-ad10-4f7e-a45a-4a154ab60ad8"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 12:52:21.652963 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.652941 5029 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-certificates\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:52:21.653132 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.653106 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb047779-ad10-4f7e-a45a-4a154ab60ad8-trusted-ca" pod "fb047779-ad10-4f7e-a45a-4a154ab60ad8" (UID: "fb047779-ad10-4f7e-a45a-4a154ab60ad8"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 12:52:21.655156 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.655132 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb047779-ad10-4f7e-a45a-4a154ab60ad8-installation-pull-secrets" pod "fb047779-ad10-4f7e-a45a-4a154ab60ad8" (UID: "fb047779-ad10-4f7e-a45a-4a154ab60ad8"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 12:52:21.655241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.655192 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-bound-sa-token" pod "fb047779-ad10-4f7e-a45a-4a154ab60ad8" (UID: "fb047779-ad10-4f7e-a45a-4a154ab60ad8"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 12:52:21.655285 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.655239 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls" pod "fb047779-ad10-4f7e-a45a-4a154ab60ad8" (UID: "fb047779-ad10-4f7e-a45a-4a154ab60ad8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 12:52:21.655498 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.655470 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb047779-ad10-4f7e-a45a-4a154ab60ad8-image-registry-private-configuration" pod "fb047779-ad10-4f7e-a45a-4a154ab60ad8" (UID: "fb047779-ad10-4f7e-a45a-4a154ab60ad8"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 12:52:21.655537 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.655496 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-kube-api-access-rk88t" pod "fb047779-ad10-4f7e-a45a-4a154ab60ad8" (UID: "fb047779-ad10-4f7e-a45a-4a154ab60ad8"). InnerVolumeSpecName "kube-api-access-rk88t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 12:52:21.660176 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.660149 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb047779-ad10-4f7e-a45a-4a154ab60ad8-ca-trust-extracted" pod "fb047779-ad10-4f7e-a45a-4a154ab60ad8" (UID: "fb047779-ad10-4f7e-a45a-4a154ab60ad8"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 12:52:21.753935 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.753903 5029 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fb047779-ad10-4f7e-a45a-4a154ab60ad8-trusted-ca\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:52:21.753935 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.753929 5029 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fb047779-ad10-4f7e-a45a-4a154ab60ad8-ca-trust-extracted\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:52:21.753935 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.753941 5029 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-registry-tls\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:52:21.754174 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.753950 5029 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fb047779-ad10-4f7e-a45a-4a154ab60ad8-installation-pull-secrets\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:52:21.754174 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.753959 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rk88t\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-kube-api-access-rk88t\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:52:21.754174 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.753968 5029 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fb047779-ad10-4f7e-a45a-4a154ab60ad8-bound-sa-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:52:21.754174 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:21.753986 5029 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fb047779-ad10-4f7e-a45a-4a154ab60ad8-image-registry-private-configuration\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:52:22.340968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:22.340884 5029 generic.go:356] "Generic (PLEG): container finished" podID="fb047779-ad10-4f7e-a45a-4a154ab60ad8" containerID="d60b37d749dd691d757bf67acf9e3902e10daf5362e1164741d9df169c2cf24e" exitCode=0 Oct 30 12:52:22.340968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:22.340954 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" Oct 30 12:52:22.340968 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:22.340963 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" event={"ID":"fb047779-ad10-4f7e-a45a-4a154ab60ad8","Type":"ContainerDied","Data":"d60b37d749dd691d757bf67acf9e3902e10daf5362e1164741d9df169c2cf24e"} Oct 30 12:52:22.341477 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:22.341041 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cfd489894-v5wlw" event={"ID":"fb047779-ad10-4f7e-a45a-4a154ab60ad8","Type":"ContainerDied","Data":"b2743e8de853dd4dc9dcc324b6986ce79b5f01cc98431abdcbb266a19408463b"} Oct 30 12:52:22.341477 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:22.341083 5029 scope.go:122] "RemoveContainer" containerID="d60b37d749dd691d757bf67acf9e3902e10daf5362e1164741d9df169c2cf24e" Oct 30 12:52:22.350234 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:22.350215 5029 scope.go:122] "RemoveContainer" containerID="d60b37d749dd691d757bf67acf9e3902e10daf5362e1164741d9df169c2cf24e" Oct 30 12:52:22.350496 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:52:22.350478 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d60b37d749dd691d757bf67acf9e3902e10daf5362e1164741d9df169c2cf24e\": container with ID starting with d60b37d749dd691d757bf67acf9e3902e10daf5362e1164741d9df169c2cf24e not found: ID does not exist" containerID="d60b37d749dd691d757bf67acf9e3902e10daf5362e1164741d9df169c2cf24e" Oct 30 12:52:22.350547 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:22.350505 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d60b37d749dd691d757bf67acf9e3902e10daf5362e1164741d9df169c2cf24e"} err="failed to get container status \"d60b37d749dd691d757bf67acf9e3902e10daf5362e1164741d9df169c2cf24e\": rpc error: code = NotFound desc = could not find container \"d60b37d749dd691d757bf67acf9e3902e10daf5362e1164741d9df169c2cf24e\": container with ID starting with d60b37d749dd691d757bf67acf9e3902e10daf5362e1164741d9df169c2cf24e not found: ID does not exist" Oct 30 12:52:22.405481 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:22.405438 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5cfd489894-v5wlw"] Oct 30 12:52:22.416653 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:22.416625 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5cfd489894-v5wlw"] Oct 30 12:52:23.087750 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:23.087717 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fb047779-ad10-4f7e-a45a-4a154ab60ad8" path="/var/lib/kubelet/pods/fb047779-ad10-4f7e-a45a-4a154ab60ad8/volumes" Oct 30 12:52:43.581758 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:43.581694 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" podUID="5133b471-c5f8-4b35-b06c-1d8f429c3ba2" containerName="registry" containerID="cri-o://b1bf434d8fcc43c9e083309fd752d13649e1b2c42efb54bba6142e0f5b723131" gracePeriod=30 Oct 30 12:52:43.817921 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:43.817898 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:52:43.940350 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:43.940273 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-bound-sa-token\") pod \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " Oct 30 12:52:43.940350 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:43.940312 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls\") pod \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " Oct 30 12:52:43.940580 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:43.940351 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-kube-api-access-gjl5n\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-kube-api-access-gjl5n\") pod \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " Oct 30 12:52:43.940580 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:43.940395 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-ca-trust-extracted\") pod \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " Oct 30 12:52:43.940580 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:43.940428 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-certificates\") pod \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " Oct 30 12:52:43.940580 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:43.940464 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-image-registry-private-configuration\") pod \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " Oct 30 12:52:43.940881 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:43.940676 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-installation-pull-secrets\") pod \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " Oct 30 12:52:43.940881 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:43.940737 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-trusted-ca\") pod \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\" (UID: \"5133b471-c5f8-4b35-b06c-1d8f429c3ba2\") " Oct 30 12:52:43.941385 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:43.941209 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-certificates" pod "5133b471-c5f8-4b35-b06c-1d8f429c3ba2" (UID: "5133b471-c5f8-4b35-b06c-1d8f429c3ba2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 12:52:43.941553 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:43.941516 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-trusted-ca" pod "5133b471-c5f8-4b35-b06c-1d8f429c3ba2" (UID: "5133b471-c5f8-4b35-b06c-1d8f429c3ba2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 12:52:43.942873 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:43.942834 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls" pod "5133b471-c5f8-4b35-b06c-1d8f429c3ba2" (UID: "5133b471-c5f8-4b35-b06c-1d8f429c3ba2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 12:52:43.943004 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:43.942880 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-bound-sa-token" pod "5133b471-c5f8-4b35-b06c-1d8f429c3ba2" (UID: "5133b471-c5f8-4b35-b06c-1d8f429c3ba2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 12:52:43.943225 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:43.943205 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-kube-api-access-gjl5n" pod "5133b471-c5f8-4b35-b06c-1d8f429c3ba2" (UID: "5133b471-c5f8-4b35-b06c-1d8f429c3ba2"). InnerVolumeSpecName "kube-api-access-gjl5n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 12:52:43.943282 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:43.943213 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-installation-pull-secrets" pod "5133b471-c5f8-4b35-b06c-1d8f429c3ba2" (UID: "5133b471-c5f8-4b35-b06c-1d8f429c3ba2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 12:52:43.943334 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:43.943318 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-image-registry-private-configuration" pod "5133b471-c5f8-4b35-b06c-1d8f429c3ba2" (UID: "5133b471-c5f8-4b35-b06c-1d8f429c3ba2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 12:52:43.947752 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:43.947727 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-ca-trust-extracted" pod "5133b471-c5f8-4b35-b06c-1d8f429c3ba2" (UID: "5133b471-c5f8-4b35-b06c-1d8f429c3ba2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 12:52:44.042241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:44.042191 5029 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-bound-sa-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:52:44.042241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:44.042234 5029 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-tls\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:52:44.042241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:44.042245 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjl5n\" (UniqueName: \"kubernetes.io/projected/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-kube-api-access-gjl5n\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:52:44.042241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:44.042256 5029 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-ca-trust-extracted\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:52:44.042502 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:44.042265 5029 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-registry-certificates\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:52:44.042502 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:44.042277 5029 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-image-registry-private-configuration\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:52:44.042502 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:44.042287 5029 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-installation-pull-secrets\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:52:44.042502 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:44.042295 5029 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5133b471-c5f8-4b35-b06c-1d8f429c3ba2-trusted-ca\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:52:44.424342 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:44.424309 5029 generic.go:356] "Generic (PLEG): container finished" podID="5133b471-c5f8-4b35-b06c-1d8f429c3ba2" containerID="b1bf434d8fcc43c9e083309fd752d13649e1b2c42efb54bba6142e0f5b723131" exitCode=0 Oct 30 12:52:44.424516 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:44.424358 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" Oct 30 12:52:44.424516 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:44.424358 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" event={"ID":"5133b471-c5f8-4b35-b06c-1d8f429c3ba2","Type":"ContainerDied","Data":"b1bf434d8fcc43c9e083309fd752d13649e1b2c42efb54bba6142e0f5b723131"} Oct 30 12:52:44.424516 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:44.424480 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cd77977d7-wrk42" event={"ID":"5133b471-c5f8-4b35-b06c-1d8f429c3ba2","Type":"ContainerDied","Data":"fcdab23e1e6361bcb45dfa5ecd318219bce98a3f25f5821342751dafb82ecfd0"} Oct 30 12:52:44.424516 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:44.424507 5029 scope.go:122] "RemoveContainer" containerID="b1bf434d8fcc43c9e083309fd752d13649e1b2c42efb54bba6142e0f5b723131" Oct 30 12:52:44.432889 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:44.432867 5029 scope.go:122] "RemoveContainer" containerID="b1bf434d8fcc43c9e083309fd752d13649e1b2c42efb54bba6142e0f5b723131" Oct 30 12:52:44.433167 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:52:44.433149 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1bf434d8fcc43c9e083309fd752d13649e1b2c42efb54bba6142e0f5b723131\": container with ID starting with b1bf434d8fcc43c9e083309fd752d13649e1b2c42efb54bba6142e0f5b723131 not found: ID does not exist" containerID="b1bf434d8fcc43c9e083309fd752d13649e1b2c42efb54bba6142e0f5b723131" Oct 30 12:52:44.433220 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:44.433177 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1bf434d8fcc43c9e083309fd752d13649e1b2c42efb54bba6142e0f5b723131"} err="failed to get container status \"b1bf434d8fcc43c9e083309fd752d13649e1b2c42efb54bba6142e0f5b723131\": rpc error: code = NotFound desc = could not find container \"b1bf434d8fcc43c9e083309fd752d13649e1b2c42efb54bba6142e0f5b723131\": container with ID starting with b1bf434d8fcc43c9e083309fd752d13649e1b2c42efb54bba6142e0f5b723131 not found: ID does not exist" Oct 30 12:52:44.492934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:44.492902 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5cd77977d7-wrk42"] Oct 30 12:52:44.498537 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:44.498515 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5cd77977d7-wrk42"] Oct 30 12:52:44.877802 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:52:44.877758 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 12:52:45.087455 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:45.087426 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5133b471-c5f8-4b35-b06c-1d8f429c3ba2" path="/var/lib/kubelet/pods/5133b471-c5f8-4b35-b06c-1d8f429c3ba2/volumes" Oct 30 12:52:50.448333 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:50.448302 5029 generic.go:356] "Generic (PLEG): container finished" podID="9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462" containerID="aed13f0d4eafb4e57ebffd1a26d0b9a89d0ab5bb01759ca9a74c5a4860ad45b5" exitCode=0 Oct 30 12:52:50.448756 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:50.448352 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-bbb6d" event={"ID":"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462","Type":"ContainerDied","Data":"aed13f0d4eafb4e57ebffd1a26d0b9a89d0ab5bb01759ca9a74c5a4860ad45b5"} Oct 30 12:52:50.448756 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:50.448659 5029 scope.go:122] "RemoveContainer" containerID="aed13f0d4eafb4e57ebffd1a26d0b9a89d0ab5bb01759ca9a74c5a4860ad45b5" Oct 30 12:52:51.044363 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.044331 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-vs88s-f66ls"] Oct 30 12:52:51.044529 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.044514 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-vs88s-f66ls" podUID="52a4a41e-8b25-4e5f-86ad-597dcd08cbd3" containerName="insights-gathering" containerID="cri-o://0e5a8081a272ead021e67e3ae5d0138c0f8a93e8814ffec411f72e6bea8de29e" gracePeriod=30 Oct 30 12:52:51.233602 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.233564 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-8c4m7-b88r5"] Oct 30 12:52:51.234133 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.234110 5029 cpu_manager.go:414] "containerMap: removing container" podUID="5133b471-c5f8-4b35-b06c-1d8f429c3ba2" containerName="registry" Oct 30 12:52:51.234133 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.234132 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5133b471-c5f8-4b35-b06c-1d8f429c3ba2" containerName="registry" Oct 30 12:52:51.234332 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.234165 5029 cpu_manager.go:414] "containerMap: removing container" podUID="fb047779-ad10-4f7e-a45a-4a154ab60ad8" containerName="registry" Oct 30 12:52:51.234332 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.234173 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb047779-ad10-4f7e-a45a-4a154ab60ad8" containerName="registry" Oct 30 12:52:51.234332 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.234259 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb047779-ad10-4f7e-a45a-4a154ab60ad8" containerName="registry" Oct 30 12:52:51.234332 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.234274 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="5133b471-c5f8-4b35-b06c-1d8f429c3ba2" containerName="registry" Oct 30 12:52:51.236671 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.236649 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-8c4m7-b88r5" Oct 30 12:52:51.256924 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.256898 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-8c4m7-b88r5"] Oct 30 12:52:51.293665 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.293643 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:52:51.297975 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.297933 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-service-ca-bundle\") pod \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " Oct 30 12:52:51.297975 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.297968 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-kube-api-access-d5fjr\" (UniqueName: \"kubernetes.io/projected/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-kube-api-access-d5fjr\") pod \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " Oct 30 12:52:51.298156 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.298032 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a574e6fd-d208-42f6-9cca-0d18f864073c-archives-path\") pod \"periodic-gathering-8c4m7-b88r5\" (UID: \"a574e6fd-d208-42f6-9cca-0d18f864073c\") " pod="openshift-insights/periodic-gathering-8c4m7-b88r5" Oct 30 12:52:51.298156 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.298097 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a574e6fd-d208-42f6-9cca-0d18f864073c-service-ca-bundle\") pod \"periodic-gathering-8c4m7-b88r5\" (UID: \"a574e6fd-d208-42f6-9cca-0d18f864073c\") " pod="openshift-insights/periodic-gathering-8c4m7-b88r5" Oct 30 12:52:51.298156 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.298122 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hn82\" (UniqueName: \"kubernetes.io/projected/a574e6fd-d208-42f6-9cca-0d18f864073c-kube-api-access-6hn82\") pod \"periodic-gathering-8c4m7-b88r5\" (UID: \"a574e6fd-d208-42f6-9cca-0d18f864073c\") " pod="openshift-insights/periodic-gathering-8c4m7-b88r5" Oct 30 12:52:51.298156 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.298146 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a574e6fd-d208-42f6-9cca-0d18f864073c-trusted-ca-bundle\") pod \"periodic-gathering-8c4m7-b88r5\" (UID: \"a574e6fd-d208-42f6-9cca-0d18f864073c\") " pod="openshift-insights/periodic-gathering-8c4m7-b88r5" Oct 30 12:52:51.298356 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.298307 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-service-ca-bundle" pod "52a4a41e-8b25-4e5f-86ad-597dcd08cbd3" (UID: "52a4a41e-8b25-4e5f-86ad-597dcd08cbd3"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 12:52:51.300084 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.300040 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-kube-api-access-d5fjr" pod "52a4a41e-8b25-4e5f-86ad-597dcd08cbd3" (UID: "52a4a41e-8b25-4e5f-86ad-597dcd08cbd3"). InnerVolumeSpecName "kube-api-access-d5fjr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 12:52:51.398999 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.398968 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-archives-path\") pod \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " Oct 30 12:52:51.399164 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.399013 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-trusted-ca-bundle\") pod \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\" (UID: \"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3\") " Oct 30 12:52:51.399164 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.399155 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a574e6fd-d208-42f6-9cca-0d18f864073c-service-ca-bundle\") pod \"periodic-gathering-8c4m7-b88r5\" (UID: \"a574e6fd-d208-42f6-9cca-0d18f864073c\") " pod="openshift-insights/periodic-gathering-8c4m7-b88r5" Oct 30 12:52:51.399292 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.399274 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hn82\" (UniqueName: \"kubernetes.io/projected/a574e6fd-d208-42f6-9cca-0d18f864073c-kube-api-access-6hn82\") pod \"periodic-gathering-8c4m7-b88r5\" (UID: \"a574e6fd-d208-42f6-9cca-0d18f864073c\") " pod="openshift-insights/periodic-gathering-8c4m7-b88r5" Oct 30 12:52:51.399349 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.399328 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a574e6fd-d208-42f6-9cca-0d18f864073c-trusted-ca-bundle\") pod \"periodic-gathering-8c4m7-b88r5\" (UID: \"a574e6fd-d208-42f6-9cca-0d18f864073c\") " pod="openshift-insights/periodic-gathering-8c4m7-b88r5" Oct 30 12:52:51.399496 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.399328 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-archives-path" pod "52a4a41e-8b25-4e5f-86ad-597dcd08cbd3" (UID: "52a4a41e-8b25-4e5f-86ad-597dcd08cbd3"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 12:52:51.399496 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.399436 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-trusted-ca-bundle" pod "52a4a41e-8b25-4e5f-86ad-597dcd08cbd3" (UID: "52a4a41e-8b25-4e5f-86ad-597dcd08cbd3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 12:52:51.399496 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.399406 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a574e6fd-d208-42f6-9cca-0d18f864073c-archives-path\") pod \"periodic-gathering-8c4m7-b88r5\" (UID: \"a574e6fd-d208-42f6-9cca-0d18f864073c\") " pod="openshift-insights/periodic-gathering-8c4m7-b88r5" Oct 30 12:52:51.399655 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.399532 5029 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-trusted-ca-bundle\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:52:51.399655 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.399548 5029 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-service-ca-bundle\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:52:51.399655 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.399558 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5fjr\" (UniqueName: \"kubernetes.io/projected/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-kube-api-access-d5fjr\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:52:51.399655 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.399568 5029 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3-archives-path\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:52:51.399809 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.399661 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a574e6fd-d208-42f6-9cca-0d18f864073c-archives-path\") pod \"periodic-gathering-8c4m7-b88r5\" (UID: \"a574e6fd-d208-42f6-9cca-0d18f864073c\") " pod="openshift-insights/periodic-gathering-8c4m7-b88r5" Oct 30 12:52:51.399936 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.399914 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a574e6fd-d208-42f6-9cca-0d18f864073c-service-ca-bundle\") pod \"periodic-gathering-8c4m7-b88r5\" (UID: \"a574e6fd-d208-42f6-9cca-0d18f864073c\") " pod="openshift-insights/periodic-gathering-8c4m7-b88r5" Oct 30 12:52:51.400001 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.399985 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a574e6fd-d208-42f6-9cca-0d18f864073c-trusted-ca-bundle\") pod \"periodic-gathering-8c4m7-b88r5\" (UID: \"a574e6fd-d208-42f6-9cca-0d18f864073c\") " pod="openshift-insights/periodic-gathering-8c4m7-b88r5" Oct 30 12:52:51.423574 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.423552 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hn82\" (UniqueName: \"kubernetes.io/projected/a574e6fd-d208-42f6-9cca-0d18f864073c-kube-api-access-6hn82\") pod \"periodic-gathering-8c4m7-b88r5\" (UID: \"a574e6fd-d208-42f6-9cca-0d18f864073c\") " pod="openshift-insights/periodic-gathering-8c4m7-b88r5" Oct 30 12:52:51.452774 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.452745 5029 generic.go:356] "Generic (PLEG): container finished" podID="52a4a41e-8b25-4e5f-86ad-597dcd08cbd3" containerID="0e5a8081a272ead021e67e3ae5d0138c0f8a93e8814ffec411f72e6bea8de29e" exitCode=2 Oct 30 12:52:51.453144 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.452822 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-vs88s-f66ls" Oct 30 12:52:51.453144 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.452845 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-vs88s-f66ls" event={"ID":"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3","Type":"ContainerDied","Data":"0e5a8081a272ead021e67e3ae5d0138c0f8a93e8814ffec411f72e6bea8de29e"} Oct 30 12:52:51.453144 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.452932 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-vs88s-f66ls" event={"ID":"52a4a41e-8b25-4e5f-86ad-597dcd08cbd3","Type":"ContainerDied","Data":"ae6e257ce29f7a6b8561fca5019d8978ad0f257f6e5e609b98d1c5d3eb418a16"} Oct 30 12:52:51.453144 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.452956 5029 scope.go:122] "RemoveContainer" containerID="0e5a8081a272ead021e67e3ae5d0138c0f8a93e8814ffec411f72e6bea8de29e" Oct 30 12:52:51.454916 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.454889 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-bbb6d" event={"ID":"9547ad8f-4eb0-48e9-a1ed-8a5efe0d5462","Type":"ContainerStarted","Data":"2444b6dbdc02ca1e73128717cef37198eaa3b823c2febc1792af06785c7d9407"} Oct 30 12:52:51.461111 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.461091 5029 scope.go:122] "RemoveContainer" containerID="0e5a8081a272ead021e67e3ae5d0138c0f8a93e8814ffec411f72e6bea8de29e" Oct 30 12:52:51.461368 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:52:51.461351 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e5a8081a272ead021e67e3ae5d0138c0f8a93e8814ffec411f72e6bea8de29e\": container with ID starting with 0e5a8081a272ead021e67e3ae5d0138c0f8a93e8814ffec411f72e6bea8de29e not found: ID does not exist" containerID="0e5a8081a272ead021e67e3ae5d0138c0f8a93e8814ffec411f72e6bea8de29e" Oct 30 12:52:51.461430 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.461375 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e5a8081a272ead021e67e3ae5d0138c0f8a93e8814ffec411f72e6bea8de29e"} err="failed to get container status \"0e5a8081a272ead021e67e3ae5d0138c0f8a93e8814ffec411f72e6bea8de29e\": rpc error: code = NotFound desc = could not find container \"0e5a8081a272ead021e67e3ae5d0138c0f8a93e8814ffec411f72e6bea8de29e\": container with ID starting with 0e5a8081a272ead021e67e3ae5d0138c0f8a93e8814ffec411f72e6bea8de29e not found: ID does not exist" Oct 30 12:52:51.503958 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.503936 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-vs88s-f66ls"] Oct 30 12:52:51.514547 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.514529 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-vs88s-f66ls"] Oct 30 12:52:51.547294 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.547275 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-8c4m7-b88r5" Oct 30 12:52:51.843542 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:51.843512 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-8c4m7-b88r5"] Oct 30 12:52:51.848368 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:52:51.848339 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda574e6fd_d208_42f6_9cca_0d18f864073c.slice/crio-cff22fd53aa04940d38184a5ce1b04de99d1f0a0ec6999a940398e6ae009211f WatchSource:0}: Error finding container cff22fd53aa04940d38184a5ce1b04de99d1f0a0ec6999a940398e6ae009211f: Status 404 returned error can't find the container with id cff22fd53aa04940d38184a5ce1b04de99d1f0a0ec6999a940398e6ae009211f Oct 30 12:52:52.460874 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:52.460836 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-8c4m7-b88r5" event={"ID":"a574e6fd-d208-42f6-9cca-0d18f864073c","Type":"ContainerStarted","Data":"62b9b1635839087cd6e1f0b619e880c06bec67aac70dfee5a380b57d61c89dde"} Oct 30 12:52:52.461368 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:52.460893 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-8c4m7-b88r5" event={"ID":"a574e6fd-d208-42f6-9cca-0d18f864073c","Type":"ContainerStarted","Data":"cff22fd53aa04940d38184a5ce1b04de99d1f0a0ec6999a940398e6ae009211f"} Oct 30 12:52:52.501531 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:52.501477 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-8c4m7-b88r5" podStartSLOduration=1.5014628989999999 podStartE2EDuration="1.501462899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:52:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:52:52.500485118 +0000 UTC m=+71.526126386" watchObservedRunningTime="2026-10-30 12:52:52.501462899 +0000 UTC m=+71.527104388" Oct 30 12:52:53.087291 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:52:53.087258 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="52a4a41e-8b25-4e5f-86ad-597dcd08cbd3" path="/var/lib/kubelet/pods/52a4a41e-8b25-4e5f-86ad-597dcd08cbd3/volumes" Oct 30 12:53:11.533528 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:53:11.533495 5029 generic.go:356] "Generic (PLEG): container finished" podID="9951ec02-ee4f-488c-8cab-3985569328f8" containerID="1a3e22b8be3970b1fec4a26737ca7d89090f863a588b21eafeea40eb71bbffcb" exitCode=0 Oct 30 12:53:11.533946 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:53:11.533539 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr" event={"ID":"9951ec02-ee4f-488c-8cab-3985569328f8","Type":"ContainerDied","Data":"1a3e22b8be3970b1fec4a26737ca7d89090f863a588b21eafeea40eb71bbffcb"} Oct 30 12:53:11.533946 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:53:11.533838 5029 scope.go:122] "RemoveContainer" containerID="1a3e22b8be3970b1fec4a26737ca7d89090f863a588b21eafeea40eb71bbffcb" Oct 30 12:53:12.539490 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:53:12.539451 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-xr7fr" event={"ID":"9951ec02-ee4f-488c-8cab-3985569328f8","Type":"ContainerStarted","Data":"07c134a62a5cd8b1ea37ec9c5a29673be2e4061a6f571b98f6141fc184d4df8a"} Oct 30 12:53:14.877661 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:53:14.877618 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 12:53:16.556023 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:53:16.555990 5029 generic.go:356] "Generic (PLEG): container finished" podID="52ca5902-69f0-462f-ae35-cbbc7f378345" containerID="c633069f0be22f8934d108cfc759dcd9d430677376e758508912027f2386eb00" exitCode=0 Oct 30 12:53:16.556444 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:53:16.556022 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp" event={"ID":"52ca5902-69f0-462f-ae35-cbbc7f378345","Type":"ContainerDied","Data":"c633069f0be22f8934d108cfc759dcd9d430677376e758508912027f2386eb00"} Oct 30 12:53:16.556444 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:53:16.556364 5029 scope.go:122] "RemoveContainer" containerID="c633069f0be22f8934d108cfc759dcd9d430677376e758508912027f2386eb00" Oct 30 12:53:17.561291 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:53:17.561250 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-fk7rp" event={"ID":"52ca5902-69f0-462f-ae35-cbbc7f378345","Type":"ContainerStarted","Data":"cff335e02d9af0795abc7400498d8cb01f166c31fa4198c38e6d15e9934466a4"} Oct 30 12:53:44.877360 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:53:44.877323 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 12:54:14.877805 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:54:14.877721 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 12:54:15.777030 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:54:15.776995 5029 generic.go:356] "Generic (PLEG): container finished" podID="a574e6fd-d208-42f6-9cca-0d18f864073c" containerID="62b9b1635839087cd6e1f0b619e880c06bec67aac70dfee5a380b57d61c89dde" exitCode=0 Oct 30 12:54:15.777219 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:54:15.777035 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-8c4m7-b88r5" event={"ID":"a574e6fd-d208-42f6-9cca-0d18f864073c","Type":"ContainerDied","Data":"62b9b1635839087cd6e1f0b619e880c06bec67aac70dfee5a380b57d61c89dde"} Oct 30 12:54:16.899913 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:54:16.899890 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-8c4m7-b88r5" Oct 30 12:54:16.962335 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:54:16.962308 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a574e6fd-d208-42f6-9cca-0d18f864073c-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a574e6fd-d208-42f6-9cca-0d18f864073c-service-ca-bundle\") pod \"a574e6fd-d208-42f6-9cca-0d18f864073c\" (UID: \"a574e6fd-d208-42f6-9cca-0d18f864073c\") " Oct 30 12:54:16.962473 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:54:16.962353 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a574e6fd-d208-42f6-9cca-0d18f864073c-kube-api-access-6hn82\" (UniqueName: \"kubernetes.io/projected/a574e6fd-d208-42f6-9cca-0d18f864073c-kube-api-access-6hn82\") pod \"a574e6fd-d208-42f6-9cca-0d18f864073c\" (UID: \"a574e6fd-d208-42f6-9cca-0d18f864073c\") " Oct 30 12:54:16.962473 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:54:16.962382 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a574e6fd-d208-42f6-9cca-0d18f864073c-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a574e6fd-d208-42f6-9cca-0d18f864073c-trusted-ca-bundle\") pod \"a574e6fd-d208-42f6-9cca-0d18f864073c\" (UID: \"a574e6fd-d208-42f6-9cca-0d18f864073c\") " Oct 30 12:54:16.962473 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:54:16.962441 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a574e6fd-d208-42f6-9cca-0d18f864073c-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a574e6fd-d208-42f6-9cca-0d18f864073c-archives-path\") pod \"a574e6fd-d208-42f6-9cca-0d18f864073c\" (UID: \"a574e6fd-d208-42f6-9cca-0d18f864073c\") " Oct 30 12:54:16.962584 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:54:16.962564 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a574e6fd-d208-42f6-9cca-0d18f864073c-service-ca-bundle" pod "a574e6fd-d208-42f6-9cca-0d18f864073c" (UID: "a574e6fd-d208-42f6-9cca-0d18f864073c"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 12:54:16.962819 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:54:16.962800 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a574e6fd-d208-42f6-9cca-0d18f864073c-archives-path" pod "a574e6fd-d208-42f6-9cca-0d18f864073c" (UID: "a574e6fd-d208-42f6-9cca-0d18f864073c"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 12:54:16.962819 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:54:16.962810 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a574e6fd-d208-42f6-9cca-0d18f864073c-trusted-ca-bundle" pod "a574e6fd-d208-42f6-9cca-0d18f864073c" (UID: "a574e6fd-d208-42f6-9cca-0d18f864073c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 12:54:16.964432 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:54:16.964405 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a574e6fd-d208-42f6-9cca-0d18f864073c-kube-api-access-6hn82" pod "a574e6fd-d208-42f6-9cca-0d18f864073c" (UID: "a574e6fd-d208-42f6-9cca-0d18f864073c"). InnerVolumeSpecName "kube-api-access-6hn82". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 12:54:17.063587 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:54:17.063555 5029 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a574e6fd-d208-42f6-9cca-0d18f864073c-service-ca-bundle\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:54:17.063587 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:54:17.063582 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hn82\" (UniqueName: \"kubernetes.io/projected/a574e6fd-d208-42f6-9cca-0d18f864073c-kube-api-access-6hn82\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:54:17.063587 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:54:17.063593 5029 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a574e6fd-d208-42f6-9cca-0d18f864073c-trusted-ca-bundle\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:54:17.063798 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:54:17.063602 5029 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a574e6fd-d208-42f6-9cca-0d18f864073c-archives-path\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:54:17.785755 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:54:17.785727 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-8c4m7-b88r5" Oct 30 12:54:17.785755 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:54:17.785736 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-8c4m7-b88r5" event={"ID":"a574e6fd-d208-42f6-9cca-0d18f864073c","Type":"ContainerDied","Data":"cff22fd53aa04940d38184a5ce1b04de99d1f0a0ec6999a940398e6ae009211f"} Oct 30 12:54:17.786030 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:54:17.785780 5029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cff22fd53aa04940d38184a5ce1b04de99d1f0a0ec6999a940398e6ae009211f" Oct 30 12:54:44.877870 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:54:44.877819 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 12:55:14.878175 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:55:14.878134 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 12:55:18.326446 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:18.326411 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-d2blh"] Oct 30 12:55:18.326920 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:18.326836 5029 cpu_manager.go:414] "containerMap: removing container" podUID="a574e6fd-d208-42f6-9cca-0d18f864073c" containerName="insights-gathering" Oct 30 12:55:18.326920 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:18.326853 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a574e6fd-d208-42f6-9cca-0d18f864073c" containerName="insights-gathering" Oct 30 12:55:18.326920 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:18.326871 5029 cpu_manager.go:414] "containerMap: removing container" podUID="52a4a41e-8b25-4e5f-86ad-597dcd08cbd3" containerName="insights-gathering" Oct 30 12:55:18.326920 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:18.326879 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="52a4a41e-8b25-4e5f-86ad-597dcd08cbd3" containerName="insights-gathering" Oct 30 12:55:18.327158 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:18.326962 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="a574e6fd-d208-42f6-9cca-0d18f864073c" containerName="insights-gathering" Oct 30 12:55:18.327158 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:18.326974 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="52a4a41e-8b25-4e5f-86ad-597dcd08cbd3" containerName="insights-gathering" Oct 30 12:55:18.329900 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:18.329880 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-d2blh" Oct 30 12:55:18.346129 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:18.346109 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Oct 30 12:55:18.356557 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:18.356535 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xklpr\"" Oct 30 12:55:18.356647 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:18.356566 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Oct 30 12:55:18.357849 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:18.357830 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-d2blh"] Oct 30 12:55:18.517559 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:18.517528 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0fba9249-3650-4dc2-ad81-9cabcb9aa4dc-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-d2blh\" (UID: \"0fba9249-3650-4dc2-ad81-9cabcb9aa4dc\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-d2blh" Oct 30 12:55:18.517723 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:18.517580 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw2lw\" (UniqueName: \"kubernetes.io/projected/0fba9249-3650-4dc2-ad81-9cabcb9aa4dc-kube-api-access-hw2lw\") pod \"cert-manager-cainjector-5c57bbd4fc-d2blh\" (UID: \"0fba9249-3650-4dc2-ad81-9cabcb9aa4dc\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-d2blh" Oct 30 12:55:18.618633 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:18.618571 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw2lw\" (UniqueName: \"kubernetes.io/projected/0fba9249-3650-4dc2-ad81-9cabcb9aa4dc-kube-api-access-hw2lw\") pod \"cert-manager-cainjector-5c57bbd4fc-d2blh\" (UID: \"0fba9249-3650-4dc2-ad81-9cabcb9aa4dc\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-d2blh" Oct 30 12:55:18.618633 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:18.618629 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0fba9249-3650-4dc2-ad81-9cabcb9aa4dc-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-d2blh\" (UID: \"0fba9249-3650-4dc2-ad81-9cabcb9aa4dc\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-d2blh" Oct 30 12:55:18.643759 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:18.643728 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0fba9249-3650-4dc2-ad81-9cabcb9aa4dc-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-d2blh\" (UID: \"0fba9249-3650-4dc2-ad81-9cabcb9aa4dc\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-d2blh" Oct 30 12:55:18.644272 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:18.644248 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw2lw\" (UniqueName: \"kubernetes.io/projected/0fba9249-3650-4dc2-ad81-9cabcb9aa4dc-kube-api-access-hw2lw\") pod \"cert-manager-cainjector-5c57bbd4fc-d2blh\" (UID: \"0fba9249-3650-4dc2-ad81-9cabcb9aa4dc\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-d2blh" Oct 30 12:55:18.654889 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:18.654866 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-d2blh" Oct 30 12:55:18.954494 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:18.954428 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-d2blh"] Oct 30 12:55:18.959851 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:55:18.959826 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0fba9249_3650_4dc2_ad81_9cabcb9aa4dc.slice/crio-7657ea9b1bb4404f1cd82d44c3bdd2faf4de60118828d8a91dde372bd8c87f0f WatchSource:0}: Error finding container 7657ea9b1bb4404f1cd82d44c3bdd2faf4de60118828d8a91dde372bd8c87f0f: Status 404 returned error can't find the container with id 7657ea9b1bb4404f1cd82d44c3bdd2faf4de60118828d8a91dde372bd8c87f0f Oct 30 12:55:19.010785 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:19.010759 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-d2blh" event={"ID":"0fba9249-3650-4dc2-ad81-9cabcb9aa4dc","Type":"ContainerStarted","Data":"7657ea9b1bb4404f1cd82d44c3bdd2faf4de60118828d8a91dde372bd8c87f0f"} Oct 30 12:55:22.025649 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:22.025564 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-d2blh" event={"ID":"0fba9249-3650-4dc2-ad81-9cabcb9aa4dc","Type":"ContainerStarted","Data":"4c1ea5e1eb0f993c13100922686f0a06fbafbd308bd6a719a0dd3303ef42df49"} Oct 30 12:55:22.066955 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:22.066896 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-d2blh" podStartSLOduration=1.169575908 podStartE2EDuration="4.066881546s" totalImagesPullingTime="2.897305638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:55:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:55:22.066294846 +0000 UTC m=+221.091936119" watchObservedRunningTime="2026-10-30 12:55:22.066881546 +0000 UTC m=+221.092522792" Oct 30 12:55:44.877989 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:55:44.877950 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 12:55:45.857951 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:45.857916 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw"] Oct 30 12:55:45.861554 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:45.861533 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" Oct 30 12:55:45.880407 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:45.880379 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Oct 30 12:55:45.891248 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:45.891227 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Oct 30 12:55:45.891248 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:45.891242 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Oct 30 12:55:45.891412 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:45.891267 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Oct 30 12:55:45.891412 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:45.891348 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-metrics-tls\"" Oct 30 12:55:45.891412 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:45.891369 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-frp85\"" Oct 30 12:55:45.918323 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:45.918299 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw"] Oct 30 12:55:45.920742 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:45.920717 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/404322af-4f87-4eb0-909e-6d2bb2cba6b5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw\" (UID: \"404322af-4f87-4eb0-909e-6d2bb2cba6b5\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" Oct 30 12:55:45.921014 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:45.920830 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/404322af-4f87-4eb0-909e-6d2bb2cba6b5-manifests\") pod \"opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw\" (UID: \"404322af-4f87-4eb0-909e-6d2bb2cba6b5\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" Oct 30 12:55:45.921014 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:45.920872 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/404322af-4f87-4eb0-909e-6d2bb2cba6b5-tmp\") pod \"opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw\" (UID: \"404322af-4f87-4eb0-909e-6d2bb2cba6b5\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" Oct 30 12:55:45.921014 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:45.920932 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/404322af-4f87-4eb0-909e-6d2bb2cba6b5-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw\" (UID: \"404322af-4f87-4eb0-909e-6d2bb2cba6b5\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" Oct 30 12:55:45.921014 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:45.920964 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcfzm\" (UniqueName: \"kubernetes.io/projected/404322af-4f87-4eb0-909e-6d2bb2cba6b5-kube-api-access-qcfzm\") pod \"opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw\" (UID: \"404322af-4f87-4eb0-909e-6d2bb2cba6b5\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" Oct 30 12:55:45.921014 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:45.921003 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/404322af-4f87-4eb0-909e-6d2bb2cba6b5-metrics-cert\") pod \"opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw\" (UID: \"404322af-4f87-4eb0-909e-6d2bb2cba6b5\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" Oct 30 12:55:46.022300 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:46.022264 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/404322af-4f87-4eb0-909e-6d2bb2cba6b5-metrics-cert\") pod \"opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw\" (UID: \"404322af-4f87-4eb0-909e-6d2bb2cba6b5\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" Oct 30 12:55:46.022300 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:46.022307 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/404322af-4f87-4eb0-909e-6d2bb2cba6b5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw\" (UID: \"404322af-4f87-4eb0-909e-6d2bb2cba6b5\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" Oct 30 12:55:46.022520 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:46.022492 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/404322af-4f87-4eb0-909e-6d2bb2cba6b5-manifests\") pod \"opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw\" (UID: \"404322af-4f87-4eb0-909e-6d2bb2cba6b5\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" Oct 30 12:55:46.022564 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:46.022533 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/404322af-4f87-4eb0-909e-6d2bb2cba6b5-tmp\") pod \"opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw\" (UID: \"404322af-4f87-4eb0-909e-6d2bb2cba6b5\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" Oct 30 12:55:46.022607 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:46.022560 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/404322af-4f87-4eb0-909e-6d2bb2cba6b5-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw\" (UID: \"404322af-4f87-4eb0-909e-6d2bb2cba6b5\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" Oct 30 12:55:46.022641 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:46.022604 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcfzm\" (UniqueName: \"kubernetes.io/projected/404322af-4f87-4eb0-909e-6d2bb2cba6b5-kube-api-access-qcfzm\") pod \"opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw\" (UID: \"404322af-4f87-4eb0-909e-6d2bb2cba6b5\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" Oct 30 12:55:46.022879 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:46.022857 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/404322af-4f87-4eb0-909e-6d2bb2cba6b5-manifests\") pod \"opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw\" (UID: \"404322af-4f87-4eb0-909e-6d2bb2cba6b5\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" Oct 30 12:55:46.024773 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:46.024746 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/404322af-4f87-4eb0-909e-6d2bb2cba6b5-tmp\") pod \"opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw\" (UID: \"404322af-4f87-4eb0-909e-6d2bb2cba6b5\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" Oct 30 12:55:46.024877 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:46.024757 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/404322af-4f87-4eb0-909e-6d2bb2cba6b5-metrics-cert\") pod \"opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw\" (UID: \"404322af-4f87-4eb0-909e-6d2bb2cba6b5\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" Oct 30 12:55:46.024932 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:46.024916 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/404322af-4f87-4eb0-909e-6d2bb2cba6b5-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw\" (UID: \"404322af-4f87-4eb0-909e-6d2bb2cba6b5\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" Oct 30 12:55:46.024965 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:46.024920 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/404322af-4f87-4eb0-909e-6d2bb2cba6b5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw\" (UID: \"404322af-4f87-4eb0-909e-6d2bb2cba6b5\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" Oct 30 12:55:46.057647 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:46.057620 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcfzm\" (UniqueName: \"kubernetes.io/projected/404322af-4f87-4eb0-909e-6d2bb2cba6b5-kube-api-access-qcfzm\") pod \"opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw\" (UID: \"404322af-4f87-4eb0-909e-6d2bb2cba6b5\") " pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" Oct 30 12:55:46.175334 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:46.175261 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" Oct 30 12:55:46.535165 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:46.535049 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw"] Oct 30 12:55:46.539628 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:55:46.539603 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod404322af_4f87_4eb0_909e_6d2bb2cba6b5.slice/crio-25f13dbc60061512f86c300311983cd01a95979f8a31065534db611b909488ea WatchSource:0}: Error finding container 25f13dbc60061512f86c300311983cd01a95979f8a31065534db611b909488ea: Status 404 returned error can't find the container with id 25f13dbc60061512f86c300311983cd01a95979f8a31065534db611b909488ea Oct 30 12:55:47.118451 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:47.118386 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" event={"ID":"404322af-4f87-4eb0-909e-6d2bb2cba6b5","Type":"ContainerStarted","Data":"25f13dbc60061512f86c300311983cd01a95979f8a31065534db611b909488ea"} Oct 30 12:55:50.134079 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:50.134043 5029 generic.go:356] "Generic (PLEG): container finished" podID="404322af-4f87-4eb0-909e-6d2bb2cba6b5" containerID="e25fa17ff777b0b765bdce2b27be2235db02da469dce6f5cb6c3e9204dc72739" exitCode=0 Oct 30 12:55:50.134470 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:50.134135 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" event={"ID":"404322af-4f87-4eb0-909e-6d2bb2cba6b5","Type":"ContainerDied","Data":"e25fa17ff777b0b765bdce2b27be2235db02da469dce6f5cb6c3e9204dc72739"} Oct 30 12:55:51.139435 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:51.139398 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" event={"ID":"404322af-4f87-4eb0-909e-6d2bb2cba6b5","Type":"ContainerStarted","Data":"8683dabf8b4a0cca3e84b9b0364336205f8cb405d04f72038c892b1d063eff30"} Oct 30 12:55:51.139919 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:51.139511 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" Oct 30 12:55:51.225720 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:55:51.225667 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" podStartSLOduration=2.465543211 podStartE2EDuration="6.225652598s" totalImagesPullingTime="3.353255777s" totalInitContainerRuntime="406.85361ms" isStatefulPod=false podCreationTimestamp="2026-10-30 12:55:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:55:51.213552732 +0000 UTC m=+250.239193982" watchObservedRunningTime="2026-10-30 12:55:51.225652598 +0000 UTC m=+250.251293921" Oct 30 12:56:02.145714 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:02.145684 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw" Oct 30 12:56:08.001763 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.001725 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-57fb7b744b-ws8l6"] Oct 30 12:56:08.005367 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.005345 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57fb7b744b-ws8l6" Oct 30 12:56:08.031734 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.031699 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57fb7b744b-ws8l6"] Oct 30 12:56:08.098348 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.098309 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7-cert\") pod \"lws-controller-manager-57fb7b744b-ws8l6\" (UID: \"1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7\") " pod="openshift-lws-operator/lws-controller-manager-57fb7b744b-ws8l6" Oct 30 12:56:08.098513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.098356 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97mh8\" (UniqueName: \"kubernetes.io/projected/1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7-kube-api-access-97mh8\") pod \"lws-controller-manager-57fb7b744b-ws8l6\" (UID: \"1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7\") " pod="openshift-lws-operator/lws-controller-manager-57fb7b744b-ws8l6" Oct 30 12:56:08.098513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.098468 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7-manager-config\") pod \"lws-controller-manager-57fb7b744b-ws8l6\" (UID: \"1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7\") " pod="openshift-lws-operator/lws-controller-manager-57fb7b744b-ws8l6" Oct 30 12:56:08.098513 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.098496 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7-metrics-cert\") pod \"lws-controller-manager-57fb7b744b-ws8l6\" (UID: \"1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7\") " pod="openshift-lws-operator/lws-controller-manager-57fb7b744b-ws8l6" Oct 30 12:56:08.109912 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.109885 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Oct 30 12:56:08.110045 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.109989 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bdndj\"" Oct 30 12:56:08.110045 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.110004 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Oct 30 12:56:08.111765 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.111751 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Oct 30 12:56:08.111808 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.111765 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Oct 30 12:56:08.111808 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.111789 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Oct 30 12:56:08.199833 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.199799 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7-cert\") pod \"lws-controller-manager-57fb7b744b-ws8l6\" (UID: \"1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7\") " pod="openshift-lws-operator/lws-controller-manager-57fb7b744b-ws8l6" Oct 30 12:56:08.200014 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.199852 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97mh8\" (UniqueName: \"kubernetes.io/projected/1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7-kube-api-access-97mh8\") pod \"lws-controller-manager-57fb7b744b-ws8l6\" (UID: \"1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7\") " pod="openshift-lws-operator/lws-controller-manager-57fb7b744b-ws8l6" Oct 30 12:56:08.200014 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.199930 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7-manager-config\") pod \"lws-controller-manager-57fb7b744b-ws8l6\" (UID: \"1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7\") " pod="openshift-lws-operator/lws-controller-manager-57fb7b744b-ws8l6" Oct 30 12:56:08.200014 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.199954 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7-metrics-cert\") pod \"lws-controller-manager-57fb7b744b-ws8l6\" (UID: \"1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7\") " pod="openshift-lws-operator/lws-controller-manager-57fb7b744b-ws8l6" Oct 30 12:56:08.200688 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.200657 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7-manager-config\") pod \"lws-controller-manager-57fb7b744b-ws8l6\" (UID: \"1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7\") " pod="openshift-lws-operator/lws-controller-manager-57fb7b744b-ws8l6" Oct 30 12:56:08.202460 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.202435 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7-cert\") pod \"lws-controller-manager-57fb7b744b-ws8l6\" (UID: \"1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7\") " pod="openshift-lws-operator/lws-controller-manager-57fb7b744b-ws8l6" Oct 30 12:56:08.202658 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.202638 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7-metrics-cert\") pod \"lws-controller-manager-57fb7b744b-ws8l6\" (UID: \"1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7\") " pod="openshift-lws-operator/lws-controller-manager-57fb7b744b-ws8l6" Oct 30 12:56:08.224600 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.224565 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97mh8\" (UniqueName: \"kubernetes.io/projected/1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7-kube-api-access-97mh8\") pod \"lws-controller-manager-57fb7b744b-ws8l6\" (UID: \"1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7\") " pod="openshift-lws-operator/lws-controller-manager-57fb7b744b-ws8l6" Oct 30 12:56:08.317231 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.317201 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57fb7b744b-ws8l6" Oct 30 12:56:08.612684 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:08.612607 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57fb7b744b-ws8l6"] Oct 30 12:56:08.617985 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:56:08.617953 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ac33bb6_b6e6_4da1_b30e_eadf4886a7d7.slice/crio-e7037a39b5307fb1cf8dd7519b2a15894afa1850aa1da433033f535374e63f30 WatchSource:0}: Error finding container e7037a39b5307fb1cf8dd7519b2a15894afa1850aa1da433033f535374e63f30: Status 404 returned error can't find the container with id e7037a39b5307fb1cf8dd7519b2a15894afa1850aa1da433033f535374e63f30 Oct 30 12:56:09.209701 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:09.209660 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57fb7b744b-ws8l6" event={"ID":"1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7","Type":"ContainerStarted","Data":"e7037a39b5307fb1cf8dd7519b2a15894afa1850aa1da433033f535374e63f30"} Oct 30 12:56:12.225734 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:12.225693 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57fb7b744b-ws8l6" event={"ID":"1ac33bb6-b6e6-4da1-b30e-eadf4886a7d7","Type":"ContainerStarted","Data":"08a1f44ba1423db77eeea7771d9f17723a55e12edae1f9099a02c82abf303c80"} Oct 30 12:56:12.226180 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:12.225792 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-57fb7b744b-ws8l6" Oct 30 12:56:12.271182 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:12.271133 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-57fb7b744b-ws8l6" podStartSLOduration=2.59143025 podStartE2EDuration="5.271117635s" totalImagesPullingTime="2.679687385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:56:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:12.270437029 +0000 UTC m=+271.296078302" watchObservedRunningTime="2026-10-30 12:56:12.271117635 +0000 UTC m=+271.296758882" Oct 30 12:56:14.879773 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:56:14.879736 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 12:56:17.124235 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.124203 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt"] Oct 30 12:56:17.127650 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.127627 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.149428 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.149404 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-h45g5\"" Oct 30 12:56:17.154682 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.154661 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Oct 30 12:56:17.156727 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.156702 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt"] Oct 30 12:56:17.275911 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.275877 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70776581-8a8f-4d3a-8454-0e6e251c3ce2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.275911 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.275910 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70776581-8a8f-4d3a-8454-0e6e251c3ce2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.276138 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.275938 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70776581-8a8f-4d3a-8454-0e6e251c3ce2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.276138 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.276008 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70776581-8a8f-4d3a-8454-0e6e251c3ce2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.276138 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.276086 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70776581-8a8f-4d3a-8454-0e6e251c3ce2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.276138 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.276120 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70776581-8a8f-4d3a-8454-0e6e251c3ce2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.276280 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.276154 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc8nh\" (UniqueName: \"kubernetes.io/projected/70776581-8a8f-4d3a-8454-0e6e251c3ce2-kube-api-access-hc8nh\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.276280 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.276229 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70776581-8a8f-4d3a-8454-0e6e251c3ce2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.276358 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.276288 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70776581-8a8f-4d3a-8454-0e6e251c3ce2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.377311 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.377226 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70776581-8a8f-4d3a-8454-0e6e251c3ce2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.377311 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.377267 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70776581-8a8f-4d3a-8454-0e6e251c3ce2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.377311 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.377294 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70776581-8a8f-4d3a-8454-0e6e251c3ce2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.377593 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.377333 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70776581-8a8f-4d3a-8454-0e6e251c3ce2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.377593 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.377358 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70776581-8a8f-4d3a-8454-0e6e251c3ce2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.377593 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.377401 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70776581-8a8f-4d3a-8454-0e6e251c3ce2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.377593 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.377427 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70776581-8a8f-4d3a-8454-0e6e251c3ce2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.377593 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.377460 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc8nh\" (UniqueName: \"kubernetes.io/projected/70776581-8a8f-4d3a-8454-0e6e251c3ce2-kube-api-access-hc8nh\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.377593 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.377493 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70776581-8a8f-4d3a-8454-0e6e251c3ce2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.378366 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.378227 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70776581-8a8f-4d3a-8454-0e6e251c3ce2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.378366 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.378308 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70776581-8a8f-4d3a-8454-0e6e251c3ce2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.378366 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.378362 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70776581-8a8f-4d3a-8454-0e6e251c3ce2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.378542 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.378474 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70776581-8a8f-4d3a-8454-0e6e251c3ce2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.378577 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.378554 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70776581-8a8f-4d3a-8454-0e6e251c3ce2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.380059 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.380034 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70776581-8a8f-4d3a-8454-0e6e251c3ce2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.380947 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.380915 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70776581-8a8f-4d3a-8454-0e6e251c3ce2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.401813 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.401790 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc8nh\" (UniqueName: \"kubernetes.io/projected/70776581-8a8f-4d3a-8454-0e6e251c3ce2-kube-api-access-hc8nh\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.401921 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.401808 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70776581-8a8f-4d3a-8454-0e6e251c3ce2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6455b4886fxpfrt\" (UID: \"70776581-8a8f-4d3a-8454-0e6e251c3ce2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.441184 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.441161 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:17.756453 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:17.756376 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt"] Oct 30 12:56:17.762395 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:56:17.762367 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70776581_8a8f_4d3a_8454_0e6e251c3ce2.slice/crio-c8ebf49fd5fa869cb4535e029e90f0111d451975b11035f9759545328f44a316 WatchSource:0}: Error finding container c8ebf49fd5fa869cb4535e029e90f0111d451975b11035f9759545328f44a316: Status 404 returned error can't find the container with id c8ebf49fd5fa869cb4535e029e90f0111d451975b11035f9759545328f44a316 Oct 30 12:56:18.249525 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:18.249489 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" event={"ID":"70776581-8a8f-4d3a-8454-0e6e251c3ce2","Type":"ContainerStarted","Data":"c8ebf49fd5fa869cb4535e029e90f0111d451975b11035f9759545328f44a316"} Oct 30 12:56:20.398453 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:20.398420 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 12:56:20.398719 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:20.398492 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 12:56:20.398719 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:20.398533 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 12:56:21.264357 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:21.264326 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" event={"ID":"70776581-8a8f-4d3a-8454-0e6e251c3ce2","Type":"ContainerStarted","Data":"a67598710f22433a628da6a01e620886af38305798a3e3fc1e23bb76e82820cd"} Oct 30 12:56:21.320357 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:21.320298 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" podStartSLOduration=1.686798075 podStartE2EDuration="4.320272984s" totalImagesPullingTime="2.633474909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:56:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:21.319690733 +0000 UTC m=+280.345331992" watchObservedRunningTime="2026-10-30 12:56:21.320272984 +0000 UTC m=+280.345914234" Oct 30 12:56:21.441746 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:21.441714 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:21.446525 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:21.446503 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:22.268121 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:22.268088 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:22.269059 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:22.269024 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886fxpfrt" Oct 30 12:56:23.232452 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:23.232419 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-57fb7b744b-ws8l6" Oct 30 12:56:23.396766 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:23.396736 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-5cfb77bf8c-b7tkl"] Oct 30 12:56:23.401801 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:23.401779 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-b7tkl" Oct 30 12:56:23.444183 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:23.444157 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-9l2gz\"" Oct 30 12:56:23.444183 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:23.444167 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\"" Oct 30 12:56:23.446208 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:23.446186 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-5cfb77bf8c-b7tkl"] Oct 30 12:56:23.531804 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:23.531731 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkqc5\" (UniqueName: \"kubernetes.io/projected/9187d211-b3d1-4945-bb8b-90593c940415-kube-api-access-kkqc5\") pod \"kserve-module-controller-manager-5cfb77bf8c-b7tkl\" (UID: \"9187d211-b3d1-4945-bb8b-90593c940415\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-b7tkl" Oct 30 12:56:23.531804 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:23.531773 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9187d211-b3d1-4945-bb8b-90593c940415-manifests\") pod \"kserve-module-controller-manager-5cfb77bf8c-b7tkl\" (UID: \"9187d211-b3d1-4945-bb8b-90593c940415\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-b7tkl" Oct 30 12:56:23.532192 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:23.531842 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9187d211-b3d1-4945-bb8b-90593c940415-tmp\") pod \"kserve-module-controller-manager-5cfb77bf8c-b7tkl\" (UID: \"9187d211-b3d1-4945-bb8b-90593c940415\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-b7tkl" Oct 30 12:56:23.532192 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:23.531878 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9187d211-b3d1-4945-bb8b-90593c940415-trusted-ca\") pod \"kserve-module-controller-manager-5cfb77bf8c-b7tkl\" (UID: \"9187d211-b3d1-4945-bb8b-90593c940415\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-b7tkl" Oct 30 12:56:23.633240 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:23.633206 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9187d211-b3d1-4945-bb8b-90593c940415-tmp\") pod \"kserve-module-controller-manager-5cfb77bf8c-b7tkl\" (UID: \"9187d211-b3d1-4945-bb8b-90593c940415\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-b7tkl" Oct 30 12:56:23.633410 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:23.633250 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9187d211-b3d1-4945-bb8b-90593c940415-trusted-ca\") pod \"kserve-module-controller-manager-5cfb77bf8c-b7tkl\" (UID: \"9187d211-b3d1-4945-bb8b-90593c940415\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-b7tkl" Oct 30 12:56:23.633410 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:23.633292 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkqc5\" (UniqueName: \"kubernetes.io/projected/9187d211-b3d1-4945-bb8b-90593c940415-kube-api-access-kkqc5\") pod \"kserve-module-controller-manager-5cfb77bf8c-b7tkl\" (UID: \"9187d211-b3d1-4945-bb8b-90593c940415\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-b7tkl" Oct 30 12:56:23.633410 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:23.633317 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9187d211-b3d1-4945-bb8b-90593c940415-manifests\") pod \"kserve-module-controller-manager-5cfb77bf8c-b7tkl\" (UID: \"9187d211-b3d1-4945-bb8b-90593c940415\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-b7tkl" Oct 30 12:56:23.633708 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:23.633681 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9187d211-b3d1-4945-bb8b-90593c940415-tmp\") pod \"kserve-module-controller-manager-5cfb77bf8c-b7tkl\" (UID: \"9187d211-b3d1-4945-bb8b-90593c940415\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-b7tkl" Oct 30 12:56:23.633784 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:23.633766 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9187d211-b3d1-4945-bb8b-90593c940415-manifests\") pod \"kserve-module-controller-manager-5cfb77bf8c-b7tkl\" (UID: \"9187d211-b3d1-4945-bb8b-90593c940415\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-b7tkl" Oct 30 12:56:23.634139 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:23.634122 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9187d211-b3d1-4945-bb8b-90593c940415-trusted-ca\") pod \"kserve-module-controller-manager-5cfb77bf8c-b7tkl\" (UID: \"9187d211-b3d1-4945-bb8b-90593c940415\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-b7tkl" Oct 30 12:56:23.664225 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:23.664194 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkqc5\" (UniqueName: \"kubernetes.io/projected/9187d211-b3d1-4945-bb8b-90593c940415-kube-api-access-kkqc5\") pod \"kserve-module-controller-manager-5cfb77bf8c-b7tkl\" (UID: \"9187d211-b3d1-4945-bb8b-90593c940415\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-b7tkl" Oct 30 12:56:23.713017 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:23.712990 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-b7tkl" Oct 30 12:56:24.053752 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:24.053720 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-5cfb77bf8c-b7tkl"] Oct 30 12:56:24.058504 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:56:24.058480 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9187d211_b3d1_4945_bb8b_90593c940415.slice/crio-b0248e71a4ed4e695c82a39d386ea195d4219ef584a0c65e2677b832d63129cb WatchSource:0}: Error finding container b0248e71a4ed4e695c82a39d386ea195d4219ef584a0c65e2677b832d63129cb: Status 404 returned error can't find the container with id b0248e71a4ed4e695c82a39d386ea195d4219ef584a0c65e2677b832d63129cb Oct 30 12:56:24.276553 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:24.276512 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-b7tkl" event={"ID":"9187d211-b3d1-4945-bb8b-90593c940415","Type":"ContainerStarted","Data":"b0248e71a4ed4e695c82a39d386ea195d4219ef584a0c65e2677b832d63129cb"} Oct 30 12:56:27.289745 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:27.289706 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-b7tkl" event={"ID":"9187d211-b3d1-4945-bb8b-90593c940415","Type":"ContainerStarted","Data":"7202b4698d89bd081b2a71056dbe1e8a9010747c541da8e9d8cd1e8f3ba01c6e"} Oct 30 12:56:27.290254 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:27.289839 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-b7tkl" Oct 30 12:56:27.345979 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:27.345859 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-b7tkl" podStartSLOduration=2.148577163 podStartE2EDuration="4.345841479s" totalImagesPullingTime="2.197264316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:56:27.344764663 +0000 UTC m=+286.370405935" watchObservedRunningTime="2026-10-30 12:56:27.345841479 +0000 UTC m=+286.371482728" Oct 30 12:56:41.106491 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:41.106460 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/2.log" Oct 30 12:56:41.106984 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:41.106744 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/2.log" Oct 30 12:56:44.878473 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:56:44.878426 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 12:56:48.295108 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:56:48.295052 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-b7tkl" Oct 30 12:57:11.133373 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.133301 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-7flkr"] Oct 30 12:57:11.145615 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.145591 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-7flkr" Oct 30 12:57:11.175219 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.175194 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-jx8p5\"" Oct 30 12:57:11.175336 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.175235 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Oct 30 12:57:11.177326 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.177299 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-7flkr"] Oct 30 12:57:11.232294 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.232265 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/218ec9c4-47f2-4fe7-82e2-f34b64920e7d-cert\") pod \"kserve-controller-manager-776cc66d47-7flkr\" (UID: \"218ec9c4-47f2-4fe7-82e2-f34b64920e7d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-7flkr" Oct 30 12:57:11.232429 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.232340 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t5jr\" (UniqueName: \"kubernetes.io/projected/218ec9c4-47f2-4fe7-82e2-f34b64920e7d-kube-api-access-2t5jr\") pod \"kserve-controller-manager-776cc66d47-7flkr\" (UID: \"218ec9c4-47f2-4fe7-82e2-f34b64920e7d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-7flkr" Oct 30 12:57:11.244229 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.244200 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww"] Oct 30 12:57:11.249361 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.249346 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww" Oct 30 12:57:11.274052 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.274027 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Oct 30 12:57:11.274052 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.274027 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-67p88\"" Oct 30 12:57:11.279372 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.279351 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww"] Oct 30 12:57:11.333301 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.333269 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcgx5\" (UniqueName: \"kubernetes.io/projected/0b0c5d07-87e4-4af6-9826-d6b6057e64c4-kube-api-access-gcgx5\") pod \"llmisvc-controller-manager-869d5dcc5d-lf7ww\" (UID: \"0b0c5d07-87e4-4af6-9826-d6b6057e64c4\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww" Oct 30 12:57:11.333434 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.333338 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b0c5d07-87e4-4af6-9826-d6b6057e64c4-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-lf7ww\" (UID: \"0b0c5d07-87e4-4af6-9826-d6b6057e64c4\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww" Oct 30 12:57:11.333434 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.333405 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t5jr\" (UniqueName: \"kubernetes.io/projected/218ec9c4-47f2-4fe7-82e2-f34b64920e7d-kube-api-access-2t5jr\") pod \"kserve-controller-manager-776cc66d47-7flkr\" (UID: \"218ec9c4-47f2-4fe7-82e2-f34b64920e7d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-7flkr" Oct 30 12:57:11.333510 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.333440 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/218ec9c4-47f2-4fe7-82e2-f34b64920e7d-cert\") pod \"kserve-controller-manager-776cc66d47-7flkr\" (UID: \"218ec9c4-47f2-4fe7-82e2-f34b64920e7d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-7flkr" Oct 30 12:57:11.335763 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.335737 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/218ec9c4-47f2-4fe7-82e2-f34b64920e7d-cert\") pod \"kserve-controller-manager-776cc66d47-7flkr\" (UID: \"218ec9c4-47f2-4fe7-82e2-f34b64920e7d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-7flkr" Oct 30 12:57:11.361834 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.361810 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t5jr\" (UniqueName: \"kubernetes.io/projected/218ec9c4-47f2-4fe7-82e2-f34b64920e7d-kube-api-access-2t5jr\") pod \"kserve-controller-manager-776cc66d47-7flkr\" (UID: \"218ec9c4-47f2-4fe7-82e2-f34b64920e7d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-7flkr" Oct 30 12:57:11.375399 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.375377 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-t2hdx"] Oct 30 12:57:11.378980 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.378963 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-t2hdx" Oct 30 12:57:11.399033 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.399010 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Oct 30 12:57:11.399148 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.399059 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-2q8hq\"" Oct 30 12:57:11.411524 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.411499 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-t2hdx"] Oct 30 12:57:11.437295 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.437265 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b0c5d07-87e4-4af6-9826-d6b6057e64c4-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-lf7ww\" (UID: \"0b0c5d07-87e4-4af6-9826-d6b6057e64c4\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww" Oct 30 12:57:11.437432 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.437334 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcgx5\" (UniqueName: \"kubernetes.io/projected/0b0c5d07-87e4-4af6-9826-d6b6057e64c4-kube-api-access-gcgx5\") pod \"llmisvc-controller-manager-869d5dcc5d-lf7ww\" (UID: \"0b0c5d07-87e4-4af6-9826-d6b6057e64c4\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww" Oct 30 12:57:11.439630 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.439607 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b0c5d07-87e4-4af6-9826-d6b6057e64c4-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-lf7ww\" (UID: \"0b0c5d07-87e4-4af6-9826-d6b6057e64c4\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww" Oct 30 12:57:11.458654 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.458632 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-7flkr" Oct 30 12:57:11.471508 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.471480 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcgx5\" (UniqueName: \"kubernetes.io/projected/0b0c5d07-87e4-4af6-9826-d6b6057e64c4-kube-api-access-gcgx5\") pod \"llmisvc-controller-manager-869d5dcc5d-lf7ww\" (UID: \"0b0c5d07-87e4-4af6-9826-d6b6057e64c4\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww" Oct 30 12:57:11.538430 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.538395 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7b8gk\" (UniqueName: \"kubernetes.io/projected/793ac5fe-0be7-457b-bb46-98c39361bbe5-kube-api-access-7b8gk\") pod \"model-serving-api-75bd89fdfc-t2hdx\" (UID: \"793ac5fe-0be7-457b-bb46-98c39361bbe5\") " pod="opendatahub/model-serving-api-75bd89fdfc-t2hdx" Oct 30 12:57:11.538651 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.538474 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/793ac5fe-0be7-457b-bb46-98c39361bbe5-tls-certs\") pod \"model-serving-api-75bd89fdfc-t2hdx\" (UID: \"793ac5fe-0be7-457b-bb46-98c39361bbe5\") " pod="opendatahub/model-serving-api-75bd89fdfc-t2hdx" Oct 30 12:57:11.563204 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.563180 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww" Oct 30 12:57:11.639728 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.639659 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/793ac5fe-0be7-457b-bb46-98c39361bbe5-tls-certs\") pod \"model-serving-api-75bd89fdfc-t2hdx\" (UID: \"793ac5fe-0be7-457b-bb46-98c39361bbe5\") " pod="opendatahub/model-serving-api-75bd89fdfc-t2hdx" Oct 30 12:57:11.639868 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.639764 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7b8gk\" (UniqueName: \"kubernetes.io/projected/793ac5fe-0be7-457b-bb46-98c39361bbe5-kube-api-access-7b8gk\") pod \"model-serving-api-75bd89fdfc-t2hdx\" (UID: \"793ac5fe-0be7-457b-bb46-98c39361bbe5\") " pod="opendatahub/model-serving-api-75bd89fdfc-t2hdx" Oct 30 12:57:11.642012 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.641990 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/793ac5fe-0be7-457b-bb46-98c39361bbe5-tls-certs\") pod \"model-serving-api-75bd89fdfc-t2hdx\" (UID: \"793ac5fe-0be7-457b-bb46-98c39361bbe5\") " pod="opendatahub/model-serving-api-75bd89fdfc-t2hdx" Oct 30 12:57:11.663849 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.663822 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7b8gk\" (UniqueName: \"kubernetes.io/projected/793ac5fe-0be7-457b-bb46-98c39361bbe5-kube-api-access-7b8gk\") pod \"model-serving-api-75bd89fdfc-t2hdx\" (UID: \"793ac5fe-0be7-457b-bb46-98c39361bbe5\") " pod="opendatahub/model-serving-api-75bd89fdfc-t2hdx" Oct 30 12:57:11.690756 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.690731 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-t2hdx" Oct 30 12:57:11.784580 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.784550 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-7flkr"] Oct 30 12:57:11.790052 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:57:11.790024 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod218ec9c4_47f2_4fe7_82e2_f34b64920e7d.slice/crio-34ae96d051d26fd89fd15049e92248427e34c4443d429d7d6c779c24e2284588 WatchSource:0}: Error finding container 34ae96d051d26fd89fd15049e92248427e34c4443d429d7d6c779c24e2284588: Status 404 returned error can't find the container with id 34ae96d051d26fd89fd15049e92248427e34c4443d429d7d6c779c24e2284588 Oct 30 12:57:11.791240 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.791223 5029 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 12:57:11.859274 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.859243 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww"] Oct 30 12:57:11.863908 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:57:11.863872 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0b0c5d07_87e4_4af6_9826_d6b6057e64c4.slice/crio-44030882304a11fc48c50de375bdef03072e769402a230c8ac5a25827c2dd549 WatchSource:0}: Error finding container 44030882304a11fc48c50de375bdef03072e769402a230c8ac5a25827c2dd549: Status 404 returned error can't find the container with id 44030882304a11fc48c50de375bdef03072e769402a230c8ac5a25827c2dd549 Oct 30 12:57:11.989088 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:11.988991 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-t2hdx"] Oct 30 12:57:11.993635 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:57:11.993603 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod793ac5fe_0be7_457b_bb46_98c39361bbe5.slice/crio-13e944627e6ebaf34d540b5415016a6c4e719cd74625d6f7ea52a976b3b4a392 WatchSource:0}: Error finding container 13e944627e6ebaf34d540b5415016a6c4e719cd74625d6f7ea52a976b3b4a392: Status 404 returned error can't find the container with id 13e944627e6ebaf34d540b5415016a6c4e719cd74625d6f7ea52a976b3b4a392 Oct 30 12:57:12.478196 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:12.478158 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-7flkr" event={"ID":"218ec9c4-47f2-4fe7-82e2-f34b64920e7d","Type":"ContainerStarted","Data":"34ae96d051d26fd89fd15049e92248427e34c4443d429d7d6c779c24e2284588"} Oct 30 12:57:12.479876 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:12.479793 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww" event={"ID":"0b0c5d07-87e4-4af6-9826-d6b6057e64c4","Type":"ContainerStarted","Data":"44030882304a11fc48c50de375bdef03072e769402a230c8ac5a25827c2dd549"} Oct 30 12:57:12.481357 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:12.481290 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-t2hdx" event={"ID":"793ac5fe-0be7-457b-bb46-98c39361bbe5","Type":"ContainerStarted","Data":"13e944627e6ebaf34d540b5415016a6c4e719cd74625d6f7ea52a976b3b4a392"} Oct 30 12:57:14.493751 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:14.493663 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-t2hdx" event={"ID":"793ac5fe-0be7-457b-bb46-98c39361bbe5","Type":"ContainerStarted","Data":"e67b66b6273c826aa4bb794e4e084c596eda691b363a310df0fae9e07bbaa1fa"} Oct 30 12:57:14.494214 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:14.493847 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-t2hdx" Oct 30 12:57:14.495304 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:14.495282 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-7flkr" event={"ID":"218ec9c4-47f2-4fe7-82e2-f34b64920e7d","Type":"ContainerStarted","Data":"78d47e272c4acb99b9ca8b8f70ee23f91d80bb14e2c05283061cb1b40224adfa"} Oct 30 12:57:14.495416 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:14.495329 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-7flkr" Oct 30 12:57:14.496827 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:14.496810 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww" event={"ID":"0b0c5d07-87e4-4af6-9826-d6b6057e64c4","Type":"ContainerStarted","Data":"533eb6d8a619b6edf32ae49b0548fa56412c8ad8794b0f427f57760c58b411fe"} Oct 30 12:57:14.496915 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:14.496856 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww" Oct 30 12:57:14.535200 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:14.535149 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-t2hdx" podStartSLOduration=1.611815252 podStartE2EDuration="3.535136208s" totalImagesPullingTime="1.923320956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:57:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:57:14.534345089 +0000 UTC m=+333.559986357" watchObservedRunningTime="2026-10-30 12:57:14.535136208 +0000 UTC m=+333.560777455" Oct 30 12:57:14.575433 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:14.575385 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww" podStartSLOduration=1.3711326910000001 podStartE2EDuration="3.57536838s" totalImagesPullingTime="2.204235689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:57:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:57:14.574680428 +0000 UTC m=+333.600321699" watchObservedRunningTime="2026-10-30 12:57:14.57536838 +0000 UTC m=+333.601009630" Oct 30 12:57:14.616778 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:14.616721 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-7flkr" podStartSLOduration=1.269664474 podStartE2EDuration="3.616706484s" totalImagesPullingTime="2.34704201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:57:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:57:14.61532606 +0000 UTC m=+333.640967330" watchObservedRunningTime="2026-10-30 12:57:14.616706484 +0000 UTC m=+333.642347732" Oct 30 12:57:14.879644 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:57:14.879596 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 12:57:25.505664 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:25.505631 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-t2hdx" Oct 30 12:57:44.877871 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:57:44.877833 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 12:57:45.503336 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:45.503305 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww" Oct 30 12:57:45.506468 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:57:45.506445 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-7flkr" Oct 30 12:58:14.878281 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:58:14.878236 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 12:58:24.634835 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:24.634754 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-gjmnr"] Oct 30 12:58:24.638248 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:24.638231 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gjmnr" Oct 30 12:58:24.648485 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:24.648463 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzrbj\" (UniqueName: \"kubernetes.io/projected/43885517-71fe-49a2-86fe-1cb4ee7fbb6b-kube-api-access-jzrbj\") pod \"kuadrant-console-plugin-66c4fb575d-gjmnr\" (UID: \"43885517-71fe-49a2-86fe-1cb4ee7fbb6b\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gjmnr" Oct 30 12:58:24.648589 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:24.648498 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/43885517-71fe-49a2-86fe-1cb4ee7fbb6b-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-gjmnr\" (UID: \"43885517-71fe-49a2-86fe-1cb4ee7fbb6b\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gjmnr" Oct 30 12:58:24.648645 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:24.648624 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/43885517-71fe-49a2-86fe-1cb4ee7fbb6b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-gjmnr\" (UID: \"43885517-71fe-49a2-86fe-1cb4ee7fbb6b\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gjmnr" Oct 30 12:58:24.659161 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:24.659124 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-gjmnr"] Oct 30 12:58:24.668676 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:24.668649 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Oct 30 12:58:24.668779 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:24.668656 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-chvlr\"" Oct 30 12:58:24.668779 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:24.668656 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Oct 30 12:58:24.668779 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:24.668657 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Oct 30 12:58:24.669301 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:24.669283 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Oct 30 12:58:24.749448 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:24.749415 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/43885517-71fe-49a2-86fe-1cb4ee7fbb6b-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-gjmnr\" (UID: \"43885517-71fe-49a2-86fe-1cb4ee7fbb6b\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gjmnr" Oct 30 12:58:24.749618 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:24.749484 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/43885517-71fe-49a2-86fe-1cb4ee7fbb6b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-gjmnr\" (UID: \"43885517-71fe-49a2-86fe-1cb4ee7fbb6b\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gjmnr" Oct 30 12:58:24.749618 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:24.749526 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzrbj\" (UniqueName: \"kubernetes.io/projected/43885517-71fe-49a2-86fe-1cb4ee7fbb6b-kube-api-access-jzrbj\") pod \"kuadrant-console-plugin-66c4fb575d-gjmnr\" (UID: \"43885517-71fe-49a2-86fe-1cb4ee7fbb6b\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gjmnr" Oct 30 12:58:24.749703 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:58:24.749630 5029 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Oct 30 12:58:24.749703 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:58:24.749698 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43885517-71fe-49a2-86fe-1cb4ee7fbb6b-plugin-serving-cert podName:43885517-71fe-49a2-86fe-1cb4ee7fbb6b nodeName:}" failed. No retries permitted until 2026-10-30 12:58:25.249679407 +0000 UTC m=+404.275320649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/43885517-71fe-49a2-86fe-1cb4ee7fbb6b-plugin-serving-cert") pod "kuadrant-console-plugin-66c4fb575d-gjmnr" (UID: "43885517-71fe-49a2-86fe-1cb4ee7fbb6b") : secret "plugin-serving-cert" not found Oct 30 12:58:24.750117 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:24.750097 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/43885517-71fe-49a2-86fe-1cb4ee7fbb6b-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-gjmnr\" (UID: \"43885517-71fe-49a2-86fe-1cb4ee7fbb6b\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gjmnr" Oct 30 12:58:24.775808 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:24.775764 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzrbj\" (UniqueName: \"kubernetes.io/projected/43885517-71fe-49a2-86fe-1cb4ee7fbb6b-kube-api-access-jzrbj\") pod \"kuadrant-console-plugin-66c4fb575d-gjmnr\" (UID: \"43885517-71fe-49a2-86fe-1cb4ee7fbb6b\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gjmnr" Oct 30 12:58:25.252457 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:25.252422 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/43885517-71fe-49a2-86fe-1cb4ee7fbb6b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-gjmnr\" (UID: \"43885517-71fe-49a2-86fe-1cb4ee7fbb6b\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gjmnr" Oct 30 12:58:25.254985 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:25.254960 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/43885517-71fe-49a2-86fe-1cb4ee7fbb6b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-gjmnr\" (UID: \"43885517-71fe-49a2-86fe-1cb4ee7fbb6b\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gjmnr" Oct 30 12:58:25.549024 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:25.548945 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gjmnr" Oct 30 12:58:25.844763 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:25.844731 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-gjmnr"] Oct 30 12:58:25.849500 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:58:25.849474 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43885517_71fe_49a2_86fe_1cb4ee7fbb6b.slice/crio-92066814a874d847d3d84127fbafced900b20d2cfeea1edf9a0e0a8b0ac5ee71 WatchSource:0}: Error finding container 92066814a874d847d3d84127fbafced900b20d2cfeea1edf9a0e0a8b0ac5ee71: Status 404 returned error can't find the container with id 92066814a874d847d3d84127fbafced900b20d2cfeea1edf9a0e0a8b0ac5ee71 Oct 30 12:58:26.795682 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:26.795603 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gjmnr" event={"ID":"43885517-71fe-49a2-86fe-1cb4ee7fbb6b","Type":"ContainerStarted","Data":"92066814a874d847d3d84127fbafced900b20d2cfeea1edf9a0e0a8b0ac5ee71"} Oct 30 12:58:30.816168 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:30.816136 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gjmnr" event={"ID":"43885517-71fe-49a2-86fe-1cb4ee7fbb6b","Type":"ContainerStarted","Data":"e021955d0813bde02840da278223eb4ca98612e999f294c6fea5b1ba73d38025"} Oct 30 12:58:30.857603 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:30.857554 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-gjmnr" podStartSLOduration=2.351634058 podStartE2EDuration="6.857540765s" totalImagesPullingTime="4.505906707s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:58:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:58:30.856776121 +0000 UTC m=+409.882417369" watchObservedRunningTime="2026-10-30 12:58:30.857540765 +0000 UTC m=+409.883182013" Oct 30 12:58:35.948206 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:35.948173 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d"] Oct 30 12:58:35.952923 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:35.952896 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d" Oct 30 12:58:35.975209 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:35.975164 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d"] Oct 30 12:58:35.980858 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:35.980824 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-t2snd\"" Oct 30 12:58:36.033059 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:36.033023 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zrg7\" (UniqueName: \"kubernetes.io/projected/81c763b8-1f11-4bd3-ac02-5a61f5cb06af-kube-api-access-2zrg7\") pod \"kuadrant-operator-controller-manager-6b6fb56846-d8l7d\" (UID: \"81c763b8-1f11-4bd3-ac02-5a61f5cb06af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d" Oct 30 12:58:36.033243 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:36.033173 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81c763b8-1f11-4bd3-ac02-5a61f5cb06af-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-d8l7d\" (UID: \"81c763b8-1f11-4bd3-ac02-5a61f5cb06af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d" Oct 30 12:58:36.133681 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:36.133649 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zrg7\" (UniqueName: \"kubernetes.io/projected/81c763b8-1f11-4bd3-ac02-5a61f5cb06af-kube-api-access-2zrg7\") pod \"kuadrant-operator-controller-manager-6b6fb56846-d8l7d\" (UID: \"81c763b8-1f11-4bd3-ac02-5a61f5cb06af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d" Oct 30 12:58:36.133843 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:36.133711 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81c763b8-1f11-4bd3-ac02-5a61f5cb06af-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-d8l7d\" (UID: \"81c763b8-1f11-4bd3-ac02-5a61f5cb06af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d" Oct 30 12:58:36.134087 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:36.134046 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81c763b8-1f11-4bd3-ac02-5a61f5cb06af-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-d8l7d\" (UID: \"81c763b8-1f11-4bd3-ac02-5a61f5cb06af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d" Oct 30 12:58:36.158964 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:36.158926 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zrg7\" (UniqueName: \"kubernetes.io/projected/81c763b8-1f11-4bd3-ac02-5a61f5cb06af-kube-api-access-2zrg7\") pod \"kuadrant-operator-controller-manager-6b6fb56846-d8l7d\" (UID: \"81c763b8-1f11-4bd3-ac02-5a61f5cb06af\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d" Oct 30 12:58:36.269295 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:36.269224 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d" Oct 30 12:58:36.619905 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:36.619862 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d"] Oct 30 12:58:36.639761 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:58:36.639722 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81c763b8_1f11_4bd3_ac02_5a61f5cb06af.slice/crio-58cae9b05ec5111306469f863c5762b76a41da25753a70da059686a98dfaddd2 WatchSource:0}: Error finding container 58cae9b05ec5111306469f863c5762b76a41da25753a70da059686a98dfaddd2: Status 404 returned error can't find the container with id 58cae9b05ec5111306469f863c5762b76a41da25753a70da059686a98dfaddd2 Oct 30 12:58:36.843304 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:36.843273 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d" event={"ID":"81c763b8-1f11-4bd3-ac02-5a61f5cb06af","Type":"ContainerStarted","Data":"58cae9b05ec5111306469f863c5762b76a41da25753a70da059686a98dfaddd2"} Oct 30 12:58:40.868906 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:40.868868 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d" event={"ID":"81c763b8-1f11-4bd3-ac02-5a61f5cb06af","Type":"ContainerStarted","Data":"99ef6cfd9f3125d36849b958736ad9028f29425cd6c0dbde1640fa9462311573"} Oct 30 12:58:40.869320 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:40.868950 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d" Oct 30 12:58:40.935484 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:40.935436 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d" podStartSLOduration=2.376679162 podStartE2EDuration="5.935419821s" totalImagesPullingTime="3.558740659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:58:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:58:40.923927798 +0000 UTC m=+419.949569066" watchObservedRunningTime="2026-10-30 12:58:40.935419821 +0000 UTC m=+419.961061065" Oct 30 12:58:44.877314 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:58:44.877278 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 12:58:50.326729 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:50.326689 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d"] Oct 30 12:58:50.327183 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:50.326904 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d" podUID="81c763b8-1f11-4bd3-ac02-5a61f5cb06af" containerName="manager" containerID="cri-o://99ef6cfd9f3125d36849b958736ad9028f29425cd6c0dbde1640fa9462311573" gracePeriod=10 Oct 30 12:58:50.328676 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:50.328651 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d" Oct 30 12:58:50.568241 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:50.568215 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d" Oct 30 12:58:50.642608 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:50.642532 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81c763b8-1f11-4bd3-ac02-5a61f5cb06af-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81c763b8-1f11-4bd3-ac02-5a61f5cb06af-extensions-socket-volume\") pod \"81c763b8-1f11-4bd3-ac02-5a61f5cb06af\" (UID: \"81c763b8-1f11-4bd3-ac02-5a61f5cb06af\") " Oct 30 12:58:50.642608 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:50.642579 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81c763b8-1f11-4bd3-ac02-5a61f5cb06af-kube-api-access-2zrg7\" (UniqueName: \"kubernetes.io/projected/81c763b8-1f11-4bd3-ac02-5a61f5cb06af-kube-api-access-2zrg7\") pod \"81c763b8-1f11-4bd3-ac02-5a61f5cb06af\" (UID: \"81c763b8-1f11-4bd3-ac02-5a61f5cb06af\") " Oct 30 12:58:50.642919 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:50.642888 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81c763b8-1f11-4bd3-ac02-5a61f5cb06af-extensions-socket-volume" pod "81c763b8-1f11-4bd3-ac02-5a61f5cb06af" (UID: "81c763b8-1f11-4bd3-ac02-5a61f5cb06af"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 12:58:50.644746 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:50.644720 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81c763b8-1f11-4bd3-ac02-5a61f5cb06af-kube-api-access-2zrg7" pod "81c763b8-1f11-4bd3-ac02-5a61f5cb06af" (UID: "81c763b8-1f11-4bd3-ac02-5a61f5cb06af"). InnerVolumeSpecName "kube-api-access-2zrg7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 12:58:50.743240 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:50.743210 5029 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/81c763b8-1f11-4bd3-ac02-5a61f5cb06af-extensions-socket-volume\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:58:50.743240 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:50.743237 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zrg7\" (UniqueName: \"kubernetes.io/projected/81c763b8-1f11-4bd3-ac02-5a61f5cb06af-kube-api-access-2zrg7\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:58:50.916601 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:50.916511 5029 generic.go:356] "Generic (PLEG): container finished" podID="81c763b8-1f11-4bd3-ac02-5a61f5cb06af" containerID="99ef6cfd9f3125d36849b958736ad9028f29425cd6c0dbde1640fa9462311573" exitCode=0 Oct 30 12:58:50.916601 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:50.916564 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d" Oct 30 12:58:50.916826 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:50.916567 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d" event={"ID":"81c763b8-1f11-4bd3-ac02-5a61f5cb06af","Type":"ContainerDied","Data":"99ef6cfd9f3125d36849b958736ad9028f29425cd6c0dbde1640fa9462311573"} Oct 30 12:58:50.916826 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:50.916684 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d" event={"ID":"81c763b8-1f11-4bd3-ac02-5a61f5cb06af","Type":"ContainerDied","Data":"58cae9b05ec5111306469f863c5762b76a41da25753a70da059686a98dfaddd2"} Oct 30 12:58:50.916826 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:50.916707 5029 scope.go:122] "RemoveContainer" containerID="99ef6cfd9f3125d36849b958736ad9028f29425cd6c0dbde1640fa9462311573" Oct 30 12:58:50.926075 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:50.926044 5029 scope.go:122] "RemoveContainer" containerID="99ef6cfd9f3125d36849b958736ad9028f29425cd6c0dbde1640fa9462311573" Oct 30 12:58:50.926342 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:58:50.926320 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99ef6cfd9f3125d36849b958736ad9028f29425cd6c0dbde1640fa9462311573\": container with ID starting with 99ef6cfd9f3125d36849b958736ad9028f29425cd6c0dbde1640fa9462311573 not found: ID does not exist" containerID="99ef6cfd9f3125d36849b958736ad9028f29425cd6c0dbde1640fa9462311573" Oct 30 12:58:50.926388 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:50.926351 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99ef6cfd9f3125d36849b958736ad9028f29425cd6c0dbde1640fa9462311573"} err="failed to get container status \"99ef6cfd9f3125d36849b958736ad9028f29425cd6c0dbde1640fa9462311573\": rpc error: code = NotFound desc = could not find container \"99ef6cfd9f3125d36849b958736ad9028f29425cd6c0dbde1640fa9462311573\": container with ID starting with 99ef6cfd9f3125d36849b958736ad9028f29425cd6c0dbde1640fa9462311573 not found: ID does not exist" Oct 30 12:58:50.979106 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:50.979051 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d"] Oct 30 12:58:50.993640 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:50.993583 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-d8l7d"] Oct 30 12:58:51.088482 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:58:51.088449 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81c763b8-1f11-4bd3-ac02-5a61f5cb06af" path="/var/lib/kubelet/pods/81c763b8-1f11-4bd3-ac02-5a61f5cb06af/volumes" Oct 30 12:59:07.511248 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.511215 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v"] Oct 30 12:59:07.511831 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.511599 5029 cpu_manager.go:414] "containerMap: removing container" podUID="81c763b8-1f11-4bd3-ac02-5a61f5cb06af" containerName="manager" Oct 30 12:59:07.511831 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.511610 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81c763b8-1f11-4bd3-ac02-5a61f5cb06af" containerName="manager" Oct 30 12:59:07.511831 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.511684 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="81c763b8-1f11-4bd3-ac02-5a61f5cb06af" containerName="manager" Oct 30 12:59:07.515999 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.515972 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.543619 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.543589 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v"] Oct 30 12:59:07.580361 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.580327 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a6dde7d-d204-46b0-b9fc-632872c1c634-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.580527 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.580371 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a6dde7d-d204-46b0-b9fc-632872c1c634-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.580527 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.580405 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a6dde7d-d204-46b0-b9fc-632872c1c634-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.580527 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.580477 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a6dde7d-d204-46b0-b9fc-632872c1c634-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.580527 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.580503 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a6dde7d-d204-46b0-b9fc-632872c1c634-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.580747 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.580568 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a6dde7d-d204-46b0-b9fc-632872c1c634-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.580747 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.580602 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a6dde7d-d204-46b0-b9fc-632872c1c634-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.580747 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.580708 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86xl6\" (UniqueName: \"kubernetes.io/projected/6a6dde7d-d204-46b0-b9fc-632872c1c634-kube-api-access-86xl6\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.580883 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.580760 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a6dde7d-d204-46b0-b9fc-632872c1c634-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.587295 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.587271 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-g8svl\"" Oct 30 12:59:07.681975 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.681936 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a6dde7d-d204-46b0-b9fc-632872c1c634-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.682177 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.681990 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a6dde7d-d204-46b0-b9fc-632872c1c634-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.682177 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.682013 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a6dde7d-d204-46b0-b9fc-632872c1c634-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.682177 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.682032 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a6dde7d-d204-46b0-b9fc-632872c1c634-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.682177 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.682048 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a6dde7d-d204-46b0-b9fc-632872c1c634-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.682177 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.682088 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a6dde7d-d204-46b0-b9fc-632872c1c634-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.682177 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.682120 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a6dde7d-d204-46b0-b9fc-632872c1c634-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.682177 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.682147 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a6dde7d-d204-46b0-b9fc-632872c1c634-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.682524 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.682198 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86xl6\" (UniqueName: \"kubernetes.io/projected/6a6dde7d-d204-46b0-b9fc-632872c1c634-kube-api-access-86xl6\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.682524 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.682469 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a6dde7d-d204-46b0-b9fc-632872c1c634-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.682631 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.682528 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a6dde7d-d204-46b0-b9fc-632872c1c634-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.682631 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.682597 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a6dde7d-d204-46b0-b9fc-632872c1c634-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.682738 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.682656 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a6dde7d-d204-46b0-b9fc-632872c1c634-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.682920 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.682897 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a6dde7d-d204-46b0-b9fc-632872c1c634-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.684391 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.684360 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a6dde7d-d204-46b0-b9fc-632872c1c634-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.684612 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.684595 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a6dde7d-d204-46b0-b9fc-632872c1c634-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.705673 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.705642 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a6dde7d-d204-46b0-b9fc-632872c1c634-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.705866 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.705837 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86xl6\" (UniqueName: \"kubernetes.io/projected/6a6dde7d-d204-46b0-b9fc-632872c1c634-kube-api-access-86xl6\") pod \"maas-default-gateway-openshift-default-5bc884dd87-lmc5v\" (UID: \"6a6dde7d-d204-46b0-b9fc-632872c1c634\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:07.829305 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:07.829272 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:08.145963 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:08.145863 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v"] Oct 30 12:59:08.151576 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:59:08.151547 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a6dde7d_d204_46b0_b9fc_632872c1c634.slice/crio-49d71c24c3d9f08efc53e1740135d64a0182ec17b4b47c31d005c62bd823f4c7 WatchSource:0}: Error finding container 49d71c24c3d9f08efc53e1740135d64a0182ec17b4b47c31d005c62bd823f4c7: Status 404 returned error can't find the container with id 49d71c24c3d9f08efc53e1740135d64a0182ec17b4b47c31d005c62bd823f4c7 Oct 30 12:59:08.153634 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:08.153602 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 12:59:08.153707 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:08.153672 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 12:59:08.153707 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:08.153702 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 12:59:08.998742 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:08.998702 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" event={"ID":"6a6dde7d-d204-46b0-b9fc-632872c1c634","Type":"ContainerStarted","Data":"ca7c6d6b3ef554f928c61b3a6ec2603b2963f92b6a26f9da2a1efcebc20620a5"} Oct 30 12:59:08.999171 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:08.998767 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" event={"ID":"6a6dde7d-d204-46b0-b9fc-632872c1c634","Type":"ContainerStarted","Data":"49d71c24c3d9f08efc53e1740135d64a0182ec17b4b47c31d005c62bd823f4c7"} Oct 30 12:59:09.055554 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:09.055503 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" podStartSLOduration=2.055489137 podStartE2EDuration="2.055489137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:59:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:59:09.054696205 +0000 UTC m=+448.080337474" watchObservedRunningTime="2026-10-30 12:59:09.055489137 +0000 UTC m=+448.081130384" Oct 30 12:59:09.830021 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:09.829988 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:09.835088 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:09.835042 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:10.003348 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:10.003318 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:10.004324 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:10.004299 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 12:59:11.879694 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:11.879652 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-9n88c"] Oct 30 12:59:11.882535 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:11.882514 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-9n88c" Oct 30 12:59:11.901534 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:11.901513 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-9n88c"] Oct 30 12:59:11.903479 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:11.903460 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-jv989\"" Oct 30 12:59:11.915618 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:11.915590 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9cxc\" (UniqueName: \"kubernetes.io/projected/c4dd8f57-432c-491d-9850-f1afa056fbf2-kube-api-access-b9cxc\") pod \"authorino-57f7f8659f-9n88c\" (UID: \"c4dd8f57-432c-491d-9850-f1afa056fbf2\") " pod="kuadrant-system/authorino-57f7f8659f-9n88c" Oct 30 12:59:12.016033 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:12.016001 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9cxc\" (UniqueName: \"kubernetes.io/projected/c4dd8f57-432c-491d-9850-f1afa056fbf2-kube-api-access-b9cxc\") pod \"authorino-57f7f8659f-9n88c\" (UID: \"c4dd8f57-432c-491d-9850-f1afa056fbf2\") " pod="kuadrant-system/authorino-57f7f8659f-9n88c" Oct 30 12:59:12.041401 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:12.041370 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9cxc\" (UniqueName: \"kubernetes.io/projected/c4dd8f57-432c-491d-9850-f1afa056fbf2-kube-api-access-b9cxc\") pod \"authorino-57f7f8659f-9n88c\" (UID: \"c4dd8f57-432c-491d-9850-f1afa056fbf2\") " pod="kuadrant-system/authorino-57f7f8659f-9n88c" Oct 30 12:59:12.193866 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:12.193776 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-9n88c" Oct 30 12:59:12.489059 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:12.488986 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-9n88c"] Oct 30 12:59:12.493927 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:59:12.493886 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4dd8f57_432c_491d_9850_f1afa056fbf2.slice/crio-50b2a870a2905495a57cd2d70467c5d784c92581273ea2995dd0b788c2bb5372 WatchSource:0}: Error finding container 50b2a870a2905495a57cd2d70467c5d784c92581273ea2995dd0b788c2bb5372: Status 404 returned error can't find the container with id 50b2a870a2905495a57cd2d70467c5d784c92581273ea2995dd0b788c2bb5372 Oct 30 12:59:12.707801 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:12.707770 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-k4s46"] Oct 30 12:59:12.710897 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:12.710877 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-k4s46" Oct 30 12:59:12.730730 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:12.730706 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-k4s46"] Oct 30 12:59:12.821928 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:12.821897 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tsm4\" (UniqueName: \"kubernetes.io/projected/e81d96c1-8169-4f96-bb3f-9267d1a1b24a-kube-api-access-2tsm4\") pod \"authorino-56c6c9ddd5-k4s46\" (UID: \"e81d96c1-8169-4f96-bb3f-9267d1a1b24a\") " pod="kuadrant-system/authorino-56c6c9ddd5-k4s46" Oct 30 12:59:12.922322 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:12.922289 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tsm4\" (UniqueName: \"kubernetes.io/projected/e81d96c1-8169-4f96-bb3f-9267d1a1b24a-kube-api-access-2tsm4\") pod \"authorino-56c6c9ddd5-k4s46\" (UID: \"e81d96c1-8169-4f96-bb3f-9267d1a1b24a\") " pod="kuadrant-system/authorino-56c6c9ddd5-k4s46" Oct 30 12:59:12.948538 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:12.948513 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tsm4\" (UniqueName: \"kubernetes.io/projected/e81d96c1-8169-4f96-bb3f-9267d1a1b24a-kube-api-access-2tsm4\") pod \"authorino-56c6c9ddd5-k4s46\" (UID: \"e81d96c1-8169-4f96-bb3f-9267d1a1b24a\") " pod="kuadrant-system/authorino-56c6c9ddd5-k4s46" Oct 30 12:59:13.021789 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:13.021759 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-k4s46" Oct 30 12:59:13.023060 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:13.023030 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-9n88c" event={"ID":"c4dd8f57-432c-491d-9850-f1afa056fbf2","Type":"ContainerStarted","Data":"50b2a870a2905495a57cd2d70467c5d784c92581273ea2995dd0b788c2bb5372"} Oct 30 12:59:13.316393 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:13.316359 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-k4s46"] Oct 30 12:59:13.321231 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:59:13.321207 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode81d96c1_8169_4f96_bb3f_9267d1a1b24a.slice/crio-8f6c331974955875fe4189bd45364d61c59ac4b9e515bd91cb8da48434988617 WatchSource:0}: Error finding container 8f6c331974955875fe4189bd45364d61c59ac4b9e515bd91cb8da48434988617: Status 404 returned error can't find the container with id 8f6c331974955875fe4189bd45364d61c59ac4b9e515bd91cb8da48434988617 Oct 30 12:59:14.029496 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:14.029458 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-k4s46" event={"ID":"e81d96c1-8169-4f96-bb3f-9267d1a1b24a","Type":"ContainerStarted","Data":"8f6c331974955875fe4189bd45364d61c59ac4b9e515bd91cb8da48434988617"} Oct 30 12:59:14.878472 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:59:14.878435 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 12:59:15.035821 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:15.035776 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-9n88c" event={"ID":"c4dd8f57-432c-491d-9850-f1afa056fbf2","Type":"ContainerStarted","Data":"cf94b629cfb4102356a85bde2761348712d177091afb943f3aec154c21c0296c"} Oct 30 12:59:15.037298 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:15.037275 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-k4s46" event={"ID":"e81d96c1-8169-4f96-bb3f-9267d1a1b24a","Type":"ContainerStarted","Data":"db288e1943702bdb6dc84c1f492e0eebf58d9e7784a31d7dee483a470d876272"} Oct 30 12:59:15.074915 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:15.074874 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-9n88c" podStartSLOduration=2.184794155 podStartE2EDuration="4.074862637s" totalImagesPullingTime="1.890068482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:59:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:59:15.073474924 +0000 UTC m=+454.099116172" watchObservedRunningTime="2026-10-30 12:59:15.074862637 +0000 UTC m=+454.100503885" Oct 30 12:59:15.113446 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:15.113395 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-k4s46" podStartSLOduration=2.046535876 podStartE2EDuration="3.113379867s" totalImagesPullingTime="1.066843991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 12:59:15.112933364 +0000 UTC m=+454.138574631" watchObservedRunningTime="2026-10-30 12:59:15.113379867 +0000 UTC m=+454.139021115" Oct 30 12:59:15.142309 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:15.142217 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-9n88c"] Oct 30 12:59:17.046378 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:17.046333 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-9n88c" podUID="c4dd8f57-432c-491d-9850-f1afa056fbf2" containerName="authorino" containerID="cri-o://cf94b629cfb4102356a85bde2761348712d177091afb943f3aec154c21c0296c" gracePeriod=30 Oct 30 12:59:17.284375 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:17.284349 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-9n88c" Oct 30 12:59:17.363046 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:17.363019 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c4dd8f57-432c-491d-9850-f1afa056fbf2-kube-api-access-b9cxc\" (UniqueName: \"kubernetes.io/projected/c4dd8f57-432c-491d-9850-f1afa056fbf2-kube-api-access-b9cxc\") pod \"c4dd8f57-432c-491d-9850-f1afa056fbf2\" (UID: \"c4dd8f57-432c-491d-9850-f1afa056fbf2\") " Oct 30 12:59:17.365047 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:17.365024 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4dd8f57-432c-491d-9850-f1afa056fbf2-kube-api-access-b9cxc" pod "c4dd8f57-432c-491d-9850-f1afa056fbf2" (UID: "c4dd8f57-432c-491d-9850-f1afa056fbf2"). InnerVolumeSpecName "kube-api-access-b9cxc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 12:59:17.463745 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:17.463713 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9cxc\" (UniqueName: \"kubernetes.io/projected/c4dd8f57-432c-491d-9850-f1afa056fbf2-kube-api-access-b9cxc\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:59:18.052567 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:18.052533 5029 generic.go:356] "Generic (PLEG): container finished" podID="c4dd8f57-432c-491d-9850-f1afa056fbf2" containerID="cf94b629cfb4102356a85bde2761348712d177091afb943f3aec154c21c0296c" exitCode=0 Oct 30 12:59:18.052966 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:18.052581 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-9n88c" Oct 30 12:59:18.052966 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:18.052590 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-9n88c" event={"ID":"c4dd8f57-432c-491d-9850-f1afa056fbf2","Type":"ContainerDied","Data":"cf94b629cfb4102356a85bde2761348712d177091afb943f3aec154c21c0296c"} Oct 30 12:59:18.052966 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:18.052630 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-9n88c" event={"ID":"c4dd8f57-432c-491d-9850-f1afa056fbf2","Type":"ContainerDied","Data":"50b2a870a2905495a57cd2d70467c5d784c92581273ea2995dd0b788c2bb5372"} Oct 30 12:59:18.052966 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:18.052651 5029 scope.go:122] "RemoveContainer" containerID="cf94b629cfb4102356a85bde2761348712d177091afb943f3aec154c21c0296c" Oct 30 12:59:18.061018 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:18.060998 5029 scope.go:122] "RemoveContainer" containerID="cf94b629cfb4102356a85bde2761348712d177091afb943f3aec154c21c0296c" Oct 30 12:59:18.061326 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:59:18.061305 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf94b629cfb4102356a85bde2761348712d177091afb943f3aec154c21c0296c\": container with ID starting with cf94b629cfb4102356a85bde2761348712d177091afb943f3aec154c21c0296c not found: ID does not exist" containerID="cf94b629cfb4102356a85bde2761348712d177091afb943f3aec154c21c0296c" Oct 30 12:59:18.061386 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:18.061337 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf94b629cfb4102356a85bde2761348712d177091afb943f3aec154c21c0296c"} err="failed to get container status \"cf94b629cfb4102356a85bde2761348712d177091afb943f3aec154c21c0296c\": rpc error: code = NotFound desc = could not find container \"cf94b629cfb4102356a85bde2761348712d177091afb943f3aec154c21c0296c\": container with ID starting with cf94b629cfb4102356a85bde2761348712d177091afb943f3aec154c21c0296c not found: ID does not exist" Oct 30 12:59:18.104643 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:18.104608 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-9n88c"] Oct 30 12:59:18.106209 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:18.106188 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-9n88c"] Oct 30 12:59:19.087292 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:19.087258 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c4dd8f57-432c-491d-9850-f1afa056fbf2" path="/var/lib/kubelet/pods/c4dd8f57-432c-491d-9850-f1afa056fbf2/volumes" Oct 30 12:59:40.286195 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:40.286159 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-tc9vs"] Oct 30 12:59:40.286657 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:40.286534 5029 cpu_manager.go:414] "containerMap: removing container" podUID="c4dd8f57-432c-491d-9850-f1afa056fbf2" containerName="authorino" Oct 30 12:59:40.286657 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:40.286546 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c4dd8f57-432c-491d-9850-f1afa056fbf2" containerName="authorino" Oct 30 12:59:40.286657 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:40.286627 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="c4dd8f57-432c-491d-9850-f1afa056fbf2" containerName="authorino" Oct 30 12:59:40.290978 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:40.290959 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-tc9vs" Oct 30 12:59:40.311353 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:40.311330 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-tc9vs"] Oct 30 12:59:40.456341 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:40.456306 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfvnb\" (UniqueName: \"kubernetes.io/projected/25830cdc-0fa0-447c-84df-eba277f322da-kube-api-access-jfvnb\") pod \"authorino-6f6944bcdd-tc9vs\" (UID: \"25830cdc-0fa0-447c-84df-eba277f322da\") " pod="kuadrant-system/authorino-6f6944bcdd-tc9vs" Oct 30 12:59:40.506527 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:40.506492 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-tc9vs"] Oct 30 12:59:40.506753 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:59:40.506729 5029 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-jfvnb], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6f6944bcdd-tc9vs" podUID="25830cdc-0fa0-447c-84df-eba277f322da" Oct 30 12:59:40.556890 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:40.556797 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfvnb\" (UniqueName: \"kubernetes.io/projected/25830cdc-0fa0-447c-84df-eba277f322da-kube-api-access-jfvnb\") pod \"authorino-6f6944bcdd-tc9vs\" (UID: \"25830cdc-0fa0-447c-84df-eba277f322da\") " pod="kuadrant-system/authorino-6f6944bcdd-tc9vs" Oct 30 12:59:40.581419 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:40.581393 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfvnb\" (UniqueName: \"kubernetes.io/projected/25830cdc-0fa0-447c-84df-eba277f322da-kube-api-access-jfvnb\") pod \"authorino-6f6944bcdd-tc9vs\" (UID: \"25830cdc-0fa0-447c-84df-eba277f322da\") " pod="kuadrant-system/authorino-6f6944bcdd-tc9vs" Oct 30 12:59:41.153148 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:41.153123 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-tc9vs" Oct 30 12:59:41.161504 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:41.161485 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-tc9vs" Oct 30 12:59:41.263418 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:41.263392 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25830cdc-0fa0-447c-84df-eba277f322da-kube-api-access-jfvnb\" (UniqueName: \"kubernetes.io/projected/25830cdc-0fa0-447c-84df-eba277f322da-kube-api-access-jfvnb\") pod \"25830cdc-0fa0-447c-84df-eba277f322da\" (UID: \"25830cdc-0fa0-447c-84df-eba277f322da\") " Oct 30 12:59:41.265428 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:41.265403 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25830cdc-0fa0-447c-84df-eba277f322da-kube-api-access-jfvnb" pod "25830cdc-0fa0-447c-84df-eba277f322da" (UID: "25830cdc-0fa0-447c-84df-eba277f322da"). InnerVolumeSpecName "kube-api-access-jfvnb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 12:59:41.364574 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:41.364500 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfvnb\" (UniqueName: \"kubernetes.io/projected/25830cdc-0fa0-447c-84df-eba277f322da-kube-api-access-jfvnb\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:59:42.156836 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:42.156805 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-tc9vs" Oct 30 12:59:42.232299 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:42.232269 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-tc9vs"] Oct 30 12:59:42.237494 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:42.237470 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-tc9vs"] Oct 30 12:59:43.087602 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:43.087568 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="25830cdc-0fa0-447c-84df-eba277f322da" path="/var/lib/kubelet/pods/25830cdc-0fa0-447c-84df-eba277f322da/volumes" Oct 30 12:59:43.420049 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:43.419966 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-k4s46"] Oct 30 12:59:43.420234 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:43.420170 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-k4s46" podUID="e81d96c1-8169-4f96-bb3f-9267d1a1b24a" containerName="authorino" containerID="cri-o://db288e1943702bdb6dc84c1f492e0eebf58d9e7784a31d7dee483a470d876272" gracePeriod=30 Oct 30 12:59:43.665228 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:43.665197 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-k4s46" Oct 30 12:59:43.784480 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:43.784388 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e81d96c1-8169-4f96-bb3f-9267d1a1b24a-kube-api-access-2tsm4\" (UniqueName: \"kubernetes.io/projected/e81d96c1-8169-4f96-bb3f-9267d1a1b24a-kube-api-access-2tsm4\") pod \"e81d96c1-8169-4f96-bb3f-9267d1a1b24a\" (UID: \"e81d96c1-8169-4f96-bb3f-9267d1a1b24a\") " Oct 30 12:59:43.786347 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:43.786321 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e81d96c1-8169-4f96-bb3f-9267d1a1b24a-kube-api-access-2tsm4" pod "e81d96c1-8169-4f96-bb3f-9267d1a1b24a" (UID: "e81d96c1-8169-4f96-bb3f-9267d1a1b24a"). InnerVolumeSpecName "kube-api-access-2tsm4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 12:59:43.885833 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:43.885800 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tsm4\" (UniqueName: \"kubernetes.io/projected/e81d96c1-8169-4f96-bb3f-9267d1a1b24a-kube-api-access-2tsm4\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 12:59:44.165991 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:44.165954 5029 generic.go:356] "Generic (PLEG): container finished" podID="e81d96c1-8169-4f96-bb3f-9267d1a1b24a" containerID="db288e1943702bdb6dc84c1f492e0eebf58d9e7784a31d7dee483a470d876272" exitCode=0 Oct 30 12:59:44.166444 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:44.165995 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-k4s46" Oct 30 12:59:44.166444 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:44.166035 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-k4s46" event={"ID":"e81d96c1-8169-4f96-bb3f-9267d1a1b24a","Type":"ContainerDied","Data":"db288e1943702bdb6dc84c1f492e0eebf58d9e7784a31d7dee483a470d876272"} Oct 30 12:59:44.166444 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:44.166089 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-k4s46" event={"ID":"e81d96c1-8169-4f96-bb3f-9267d1a1b24a","Type":"ContainerDied","Data":"8f6c331974955875fe4189bd45364d61c59ac4b9e515bd91cb8da48434988617"} Oct 30 12:59:44.166444 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:44.166106 5029 scope.go:122] "RemoveContainer" containerID="db288e1943702bdb6dc84c1f492e0eebf58d9e7784a31d7dee483a470d876272" Oct 30 12:59:44.174924 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:44.174907 5029 scope.go:122] "RemoveContainer" containerID="db288e1943702bdb6dc84c1f492e0eebf58d9e7784a31d7dee483a470d876272" Oct 30 12:59:44.175185 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:59:44.175165 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db288e1943702bdb6dc84c1f492e0eebf58d9e7784a31d7dee483a470d876272\": container with ID starting with db288e1943702bdb6dc84c1f492e0eebf58d9e7784a31d7dee483a470d876272 not found: ID does not exist" containerID="db288e1943702bdb6dc84c1f492e0eebf58d9e7784a31d7dee483a470d876272" Oct 30 12:59:44.175236 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:44.175194 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db288e1943702bdb6dc84c1f492e0eebf58d9e7784a31d7dee483a470d876272"} err="failed to get container status \"db288e1943702bdb6dc84c1f492e0eebf58d9e7784a31d7dee483a470d876272\": rpc error: code = NotFound desc = could not find container \"db288e1943702bdb6dc84c1f492e0eebf58d9e7784a31d7dee483a470d876272\": container with ID starting with db288e1943702bdb6dc84c1f492e0eebf58d9e7784a31d7dee483a470d876272 not found: ID does not exist" Oct 30 12:59:44.219919 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:44.219892 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-k4s46"] Oct 30 12:59:44.231270 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:44.231250 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-k4s46"] Oct 30 12:59:44.878042 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:59:44.878008 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 12:59:45.088823 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:45.088782 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e81d96c1-8169-4f96-bb3f-9267d1a1b24a" path="/var/lib/kubelet/pods/e81d96c1-8169-4f96-bb3f-9267d1a1b24a/volumes" Oct 30 12:59:57.750797 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:57.750707 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6"] Oct 30 12:59:57.751182 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:57.751088 5029 cpu_manager.go:414] "containerMap: removing container" podUID="e81d96c1-8169-4f96-bb3f-9267d1a1b24a" containerName="authorino" Oct 30 12:59:57.751182 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:57.751102 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e81d96c1-8169-4f96-bb3f-9267d1a1b24a" containerName="authorino" Oct 30 12:59:57.751265 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:57.751187 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="e81d96c1-8169-4f96-bb3f-9267d1a1b24a" containerName="authorino" Oct 30 12:59:57.754088 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:57.754059 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6" Oct 30 12:59:57.770279 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:57.770256 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\"" Oct 30 12:59:57.782171 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:57.782148 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\"" Oct 30 12:59:57.782300 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:57.782183 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\"" Oct 30 12:59:57.785934 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:57.785905 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6"] Oct 30 12:59:57.794051 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:57.794028 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmd7g\" (UniqueName: \"kubernetes.io/projected/5029dd68-f246-4a21-8181-f5a5dbb9f574-kube-api-access-wmd7g\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jr9k6\" (UID: \"5029dd68-f246-4a21-8181-f5a5dbb9f574\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6" Oct 30 12:59:57.794170 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:57.794088 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5029dd68-f246-4a21-8181-f5a5dbb9f574-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jr9k6\" (UID: \"5029dd68-f246-4a21-8181-f5a5dbb9f574\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6" Oct 30 12:59:57.794242 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:57.794221 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5029dd68-f246-4a21-8181-f5a5dbb9f574-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jr9k6\" (UID: \"5029dd68-f246-4a21-8181-f5a5dbb9f574\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6" Oct 30 12:59:57.794310 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:57.794298 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/5029dd68-f246-4a21-8181-f5a5dbb9f574-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jr9k6\" (UID: \"5029dd68-f246-4a21-8181-f5a5dbb9f574\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6" Oct 30 12:59:57.811195 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:57.811174 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-cmgcz\"" Oct 30 12:59:57.894700 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:57.894665 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5029dd68-f246-4a21-8181-f5a5dbb9f574-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jr9k6\" (UID: \"5029dd68-f246-4a21-8181-f5a5dbb9f574\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6" Oct 30 12:59:57.894883 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:57.894720 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/5029dd68-f246-4a21-8181-f5a5dbb9f574-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jr9k6\" (UID: \"5029dd68-f246-4a21-8181-f5a5dbb9f574\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6" Oct 30 12:59:57.894883 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:57.894794 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmd7g\" (UniqueName: \"kubernetes.io/projected/5029dd68-f246-4a21-8181-f5a5dbb9f574-kube-api-access-wmd7g\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jr9k6\" (UID: \"5029dd68-f246-4a21-8181-f5a5dbb9f574\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6" Oct 30 12:59:57.894883 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:59:57.894858 5029 configmap.go:193] Couldn't get configMap opendatahub/usage-tenancy-proxy-source: configmap "usage-tenancy-proxy-source" not found Oct 30 12:59:57.895039 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:57.894899 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5029dd68-f246-4a21-8181-f5a5dbb9f574-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jr9k6\" (UID: \"5029dd68-f246-4a21-8181-f5a5dbb9f574\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6" Oct 30 12:59:57.895039 ip-10-0-141-147 kubenswrapper[5029]: E1030 12:59:57.894934 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5029dd68-f246-4a21-8181-f5a5dbb9f574-proxy-source podName:5029dd68-f246-4a21-8181-f5a5dbb9f574 nodeName:}" failed. No retries permitted until 2026-10-30 12:59:58.394918964 +0000 UTC m=+497.420560212 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-source" (UniqueName: "kubernetes.io/configmap/5029dd68-f246-4a21-8181-f5a5dbb9f574-proxy-source") pod "usage-logs-tenancy-proxy-7d94f55f4b-jr9k6" (UID: "5029dd68-f246-4a21-8181-f5a5dbb9f574") : configmap "usage-tenancy-proxy-source" not found Oct 30 12:59:57.895526 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:57.895507 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5029dd68-f246-4a21-8181-f5a5dbb9f574-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jr9k6\" (UID: \"5029dd68-f246-4a21-8181-f5a5dbb9f574\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6" Oct 30 12:59:57.897296 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:57.897278 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5029dd68-f246-4a21-8181-f5a5dbb9f574-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jr9k6\" (UID: \"5029dd68-f246-4a21-8181-f5a5dbb9f574\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6" Oct 30 12:59:57.921143 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:57.921112 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmd7g\" (UniqueName: \"kubernetes.io/projected/5029dd68-f246-4a21-8181-f5a5dbb9f574-kube-api-access-wmd7g\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jr9k6\" (UID: \"5029dd68-f246-4a21-8181-f5a5dbb9f574\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6" Oct 30 12:59:58.398782 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:58.398743 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/5029dd68-f246-4a21-8181-f5a5dbb9f574-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jr9k6\" (UID: \"5029dd68-f246-4a21-8181-f5a5dbb9f574\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6" Oct 30 12:59:58.399454 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:58.399436 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/5029dd68-f246-4a21-8181-f5a5dbb9f574-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-jr9k6\" (UID: \"5029dd68-f246-4a21-8181-f5a5dbb9f574\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6" Oct 30 12:59:58.666752 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:58.666670 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6" Oct 30 12:59:58.964349 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:58.964262 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6"] Oct 30 12:59:58.968480 ip-10-0-141-147 kubenswrapper[5029]: W1030 12:59:58.968454 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5029dd68_f246_4a21_8181_f5a5dbb9f574.slice/crio-85b7788de7d932702ec455d2afdd875206e039715240dcea0da526fe51a45089 WatchSource:0}: Error finding container 85b7788de7d932702ec455d2afdd875206e039715240dcea0da526fe51a45089: Status 404 returned error can't find the container with id 85b7788de7d932702ec455d2afdd875206e039715240dcea0da526fe51a45089 Oct 30 12:59:59.229481 ip-10-0-141-147 kubenswrapper[5029]: I1030 12:59:59.229395 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6" event={"ID":"5029dd68-f246-4a21-8181-f5a5dbb9f574","Type":"ContainerStarted","Data":"85b7788de7d932702ec455d2afdd875206e039715240dcea0da526fe51a45089"} Oct 30 13:00:09.281635 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:09.281594 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6" event={"ID":"5029dd68-f246-4a21-8181-f5a5dbb9f574","Type":"ContainerStarted","Data":"e310c5b61a4d801a0c1f9f73389814d3622e21349633a06858e3708b7ea28afa"} Oct 30 13:00:09.282014 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:09.281656 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6" Oct 30 13:00:09.323421 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:09.323374 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6" podStartSLOduration=2.490468595 podStartE2EDuration="12.323358803s" totalImagesPullingTime="9.832890208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 12:59:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:00:09.322854089 +0000 UTC m=+508.348495337" watchObservedRunningTime="2026-10-30 13:00:09.323358803 +0000 UTC m=+508.349000053" Oct 30 13:00:14.877360 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:00:14.877322 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:00:20.291749 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:20.291712 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-jr9k6" Oct 30 13:00:27.116330 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:27.116296 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx"] Oct 30 13:00:27.119630 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:27.119610 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" Oct 30 13:00:27.148279 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:27.148252 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 13:00:27.148478 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:27.148257 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-d5n24\"" Oct 30 13:00:27.148478 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:27.148331 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Oct 30 13:00:27.148668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:27.148565 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 13:00:27.150134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:27.150113 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx"] Oct 30 13:00:27.237744 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:27.237720 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Oct 30 13:00:27.249103 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:27.249062 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc5mh\" (UniqueName: \"kubernetes.io/projected/89ee7c56-9d24-468e-a804-8729861ad1bd-kube-api-access-jc5mh\") pod \"maas-api-5c48877c75-zg7sx\" (UID: \"89ee7c56-9d24-468e-a804-8729861ad1bd\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" Oct 30 13:00:27.249243 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:27.249187 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89ee7c56-9d24-468e-a804-8729861ad1bd-openshift-service-ca\") pod \"maas-api-5c48877c75-zg7sx\" (UID: \"89ee7c56-9d24-468e-a804-8729861ad1bd\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" Oct 30 13:00:27.249243 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:27.249218 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ee7c56-9d24-468e-a804-8729861ad1bd-metrics-certs\") pod \"maas-api-5c48877c75-zg7sx\" (UID: \"89ee7c56-9d24-468e-a804-8729861ad1bd\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" Oct 30 13:00:27.249356 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:27.249297 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89ee7c56-9d24-468e-a804-8729861ad1bd-maas-api-tls\") pod \"maas-api-5c48877c75-zg7sx\" (UID: \"89ee7c56-9d24-468e-a804-8729861ad1bd\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" Oct 30 13:00:27.350238 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:27.350207 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89ee7c56-9d24-468e-a804-8729861ad1bd-openshift-service-ca\") pod \"maas-api-5c48877c75-zg7sx\" (UID: \"89ee7c56-9d24-468e-a804-8729861ad1bd\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" Oct 30 13:00:27.350391 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:27.350253 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ee7c56-9d24-468e-a804-8729861ad1bd-metrics-certs\") pod \"maas-api-5c48877c75-zg7sx\" (UID: \"89ee7c56-9d24-468e-a804-8729861ad1bd\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" Oct 30 13:00:27.350391 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:27.350289 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89ee7c56-9d24-468e-a804-8729861ad1bd-maas-api-tls\") pod \"maas-api-5c48877c75-zg7sx\" (UID: \"89ee7c56-9d24-468e-a804-8729861ad1bd\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" Oct 30 13:00:27.350391 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:27.350313 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc5mh\" (UniqueName: \"kubernetes.io/projected/89ee7c56-9d24-468e-a804-8729861ad1bd-kube-api-access-jc5mh\") pod \"maas-api-5c48877c75-zg7sx\" (UID: \"89ee7c56-9d24-468e-a804-8729861ad1bd\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" Oct 30 13:00:27.350834 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:27.350809 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89ee7c56-9d24-468e-a804-8729861ad1bd-openshift-service-ca\") pod \"maas-api-5c48877c75-zg7sx\" (UID: \"89ee7c56-9d24-468e-a804-8729861ad1bd\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" Oct 30 13:00:27.352727 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:27.352704 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89ee7c56-9d24-468e-a804-8729861ad1bd-maas-api-tls\") pod \"maas-api-5c48877c75-zg7sx\" (UID: \"89ee7c56-9d24-468e-a804-8729861ad1bd\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" Oct 30 13:00:27.352912 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:27.352891 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ee7c56-9d24-468e-a804-8729861ad1bd-metrics-certs\") pod \"maas-api-5c48877c75-zg7sx\" (UID: \"89ee7c56-9d24-468e-a804-8729861ad1bd\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" Oct 30 13:00:27.374423 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:27.374360 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc5mh\" (UniqueName: \"kubernetes.io/projected/89ee7c56-9d24-468e-a804-8729861ad1bd-kube-api-access-jc5mh\") pod \"maas-api-5c48877c75-zg7sx\" (UID: \"89ee7c56-9d24-468e-a804-8729861ad1bd\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" Oct 30 13:00:27.432311 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:27.432283 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" Oct 30 13:00:27.735943 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:27.735861 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx"] Oct 30 13:00:27.740797 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:00:27.740762 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89ee7c56_9d24_468e_a804_8729861ad1bd.slice/crio-048d0fe92b9b407fae5e6c8bb01d89abd0302c179a9641a07737f30dd764f47d WatchSource:0}: Error finding container 048d0fe92b9b407fae5e6c8bb01d89abd0302c179a9641a07737f30dd764f47d: Status 404 returned error can't find the container with id 048d0fe92b9b407fae5e6c8bb01d89abd0302c179a9641a07737f30dd764f47d Oct 30 13:00:28.373692 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:28.373653 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" event={"ID":"89ee7c56-9d24-468e-a804-8729861ad1bd","Type":"ContainerStarted","Data":"048d0fe92b9b407fae5e6c8bb01d89abd0302c179a9641a07737f30dd764f47d"} Oct 30 13:00:30.400125 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:30.400012 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" event={"ID":"89ee7c56-9d24-468e-a804-8729861ad1bd","Type":"ContainerStarted","Data":"d5baca55c310c9b051b2fc8be94dfc10cbfa89022c2e029471a56e3ce252a354"} Oct 30 13:00:30.400125 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:30.400120 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" Oct 30 13:00:30.442562 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:30.442517 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" podStartSLOduration=1.049153292 podStartE2EDuration="3.442503098s" totalImagesPullingTime="2.393349806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:00:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:00:30.441440885 +0000 UTC m=+529.467082152" watchObservedRunningTime="2026-10-30 13:00:30.442503098 +0000 UTC m=+529.468144346" Oct 30 13:00:36.410668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:36.410630 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" Oct 30 13:00:37.513708 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:37.513671 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-c27pr"] Oct 30 13:00:37.802250 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:37.802168 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-c27pr"] Oct 30 13:00:37.802391 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:37.802286 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-c27pr" Oct 30 13:00:37.967710 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:37.967664 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa264999-e5bc-4ff5-83c8-0cc23321891c-cert\") pod \"llmisvc-controller-manager-6466c554d8-c27pr\" (UID: \"aa264999-e5bc-4ff5-83c8-0cc23321891c\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-c27pr" Oct 30 13:00:37.967880 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:37.967827 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmj7b\" (UniqueName: \"kubernetes.io/projected/aa264999-e5bc-4ff5-83c8-0cc23321891c-kube-api-access-pmj7b\") pod \"llmisvc-controller-manager-6466c554d8-c27pr\" (UID: \"aa264999-e5bc-4ff5-83c8-0cc23321891c\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-c27pr" Oct 30 13:00:38.068315 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:38.068282 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmj7b\" (UniqueName: \"kubernetes.io/projected/aa264999-e5bc-4ff5-83c8-0cc23321891c-kube-api-access-pmj7b\") pod \"llmisvc-controller-manager-6466c554d8-c27pr\" (UID: \"aa264999-e5bc-4ff5-83c8-0cc23321891c\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-c27pr" Oct 30 13:00:38.068491 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:38.068333 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa264999-e5bc-4ff5-83c8-0cc23321891c-cert\") pod \"llmisvc-controller-manager-6466c554d8-c27pr\" (UID: \"aa264999-e5bc-4ff5-83c8-0cc23321891c\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-c27pr" Oct 30 13:00:38.070756 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:38.070731 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aa264999-e5bc-4ff5-83c8-0cc23321891c-cert\") pod \"llmisvc-controller-manager-6466c554d8-c27pr\" (UID: \"aa264999-e5bc-4ff5-83c8-0cc23321891c\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-c27pr" Oct 30 13:00:38.092726 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:38.092696 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmj7b\" (UniqueName: \"kubernetes.io/projected/aa264999-e5bc-4ff5-83c8-0cc23321891c-kube-api-access-pmj7b\") pod \"llmisvc-controller-manager-6466c554d8-c27pr\" (UID: \"aa264999-e5bc-4ff5-83c8-0cc23321891c\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-c27pr" Oct 30 13:00:38.138586 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:38.138552 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-c27pr" Oct 30 13:00:38.632813 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:38.572197 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-c27pr"] Oct 30 13:00:38.632813 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:00:38.576307 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa264999_e5bc_4ff5_83c8_0cc23321891c.slice/crio-7dd701f7deb273f0472d2557a1311765a43002474c0dc67f03e7fa2c29116cd8 WatchSource:0}: Error finding container 7dd701f7deb273f0472d2557a1311765a43002474c0dc67f03e7fa2c29116cd8: Status 404 returned error can't find the container with id 7dd701f7deb273f0472d2557a1311765a43002474c0dc67f03e7fa2c29116cd8 Oct 30 13:00:39.440480 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:39.440448 5029 generic.go:356] "Generic (PLEG): container finished" podID="6a6dde7d-d204-46b0-b9fc-632872c1c634" containerID="ca7c6d6b3ef554f928c61b3a6ec2603b2963f92b6a26f9da2a1efcebc20620a5" exitCode=0 Oct 30 13:00:39.440659 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:39.440526 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" event={"ID":"6a6dde7d-d204-46b0-b9fc-632872c1c634","Type":"ContainerDied","Data":"ca7c6d6b3ef554f928c61b3a6ec2603b2963f92b6a26f9da2a1efcebc20620a5"} Oct 30 13:00:39.440944 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:39.440919 5029 scope.go:122] "RemoveContainer" containerID="ca7c6d6b3ef554f928c61b3a6ec2603b2963f92b6a26f9da2a1efcebc20620a5" Oct 30 13:00:39.442217 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:39.442181 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:00:39.442338 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:39.442264 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:00:39.442338 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:39.442309 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:00:39.442439 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:39.442411 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-c27pr" event={"ID":"aa264999-e5bc-4ff5-83c8-0cc23321891c","Type":"ContainerStarted","Data":"cca8610e37309cfe936979ee745cd1e08b3182888643287bbb9998ee7292d1c3"} Oct 30 13:00:39.442493 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:39.442449 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-c27pr" event={"ID":"aa264999-e5bc-4ff5-83c8-0cc23321891c","Type":"ContainerStarted","Data":"7dd701f7deb273f0472d2557a1311765a43002474c0dc67f03e7fa2c29116cd8"} Oct 30 13:00:39.442550 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:39.442512 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-c27pr" Oct 30 13:00:39.539006 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:39.538949 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-c27pr" podStartSLOduration=2.057799664 podStartE2EDuration="2.538930026s" totalImagesPullingTime="481.130362ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:00:39.536830844 +0000 UTC m=+538.562472093" watchObservedRunningTime="2026-10-30 13:00:39.538930026 +0000 UTC m=+538.564571276" Oct 30 13:00:40.004116 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:40.004082 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 13:00:40.449334 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:40.449293 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" event={"ID":"6a6dde7d-d204-46b0-b9fc-632872c1c634","Type":"ContainerStarted","Data":"6aaa3e01ce11993b389aab06643295528ceb1728734e6a2027a9a7c5077f475f"} Oct 30 13:00:40.829876 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:40.829845 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 13:00:40.831095 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:40.831047 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-lmc5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Oct 30 13:00:40.831205 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:40.831121 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" podUID="6a6dde7d-d204-46b0-b9fc-632872c1c634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Oct 30 13:00:41.829833 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:41.829797 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-lmc5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Oct 30 13:00:41.830218 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:41.829855 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" podUID="6a6dde7d-d204-46b0-b9fc-632872c1c634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Oct 30 13:00:42.830345 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:42.830308 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-lmc5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Oct 30 13:00:42.830748 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:42.830368 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" podUID="6a6dde7d-d204-46b0-b9fc-632872c1c634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Oct 30 13:00:43.829623 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:43.829583 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-lmc5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Oct 30 13:00:43.829800 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:43.829650 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" podUID="6a6dde7d-d204-46b0-b9fc-632872c1c634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Oct 30 13:00:44.830342 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:44.830304 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-lmc5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Oct 30 13:00:44.830957 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:44.830365 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" podUID="6a6dde7d-d204-46b0-b9fc-632872c1c634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Oct 30 13:00:44.902460 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:00:44.902426 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:00:45.833657 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:45.833626 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 13:00:45.834047 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:45.833907 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 13:00:45.834577 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:00:45.834561 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" Oct 30 13:01:10.451639 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:01:10.451608 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-c27pr" Oct 30 13:01:10.524171 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:01:10.524132 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww"] Oct 30 13:01:10.524505 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:01:10.524459 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww" podUID="0b0c5d07-87e4-4af6-9826-d6b6057e64c4" containerName="manager" containerID="cri-o://533eb6d8a619b6edf32ae49b0548fa56412c8ad8794b0f427f57760c58b411fe" gracePeriod=30 Oct 30 13:01:10.770548 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:01:10.770524 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww" Oct 30 13:01:10.841797 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:01:10.841769 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0b0c5d07-87e4-4af6-9826-d6b6057e64c4-cert\" (UniqueName: \"kubernetes.io/secret/0b0c5d07-87e4-4af6-9826-d6b6057e64c4-cert\") pod \"0b0c5d07-87e4-4af6-9826-d6b6057e64c4\" (UID: \"0b0c5d07-87e4-4af6-9826-d6b6057e64c4\") " Oct 30 13:01:10.841960 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:01:10.841910 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b0c5d07-87e4-4af6-9826-d6b6057e64c4-kube-api-access-gcgx5\" (UniqueName: \"kubernetes.io/projected/0b0c5d07-87e4-4af6-9826-d6b6057e64c4-kube-api-access-gcgx5\") pod \"0b0c5d07-87e4-4af6-9826-d6b6057e64c4\" (UID: \"0b0c5d07-87e4-4af6-9826-d6b6057e64c4\") " Oct 30 13:01:10.843985 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:01:10.843957 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b0c5d07-87e4-4af6-9826-d6b6057e64c4-cert" pod "0b0c5d07-87e4-4af6-9826-d6b6057e64c4" (UID: "0b0c5d07-87e4-4af6-9826-d6b6057e64c4"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:01:10.844112 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:01:10.843981 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b0c5d07-87e4-4af6-9826-d6b6057e64c4-kube-api-access-gcgx5" pod "0b0c5d07-87e4-4af6-9826-d6b6057e64c4" (UID: "0b0c5d07-87e4-4af6-9826-d6b6057e64c4"). InnerVolumeSpecName "kube-api-access-gcgx5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:01:10.942526 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:01:10.942493 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gcgx5\" (UniqueName: \"kubernetes.io/projected/0b0c5d07-87e4-4af6-9826-d6b6057e64c4-kube-api-access-gcgx5\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:01:10.942526 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:01:10.942522 5029 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0b0c5d07-87e4-4af6-9826-d6b6057e64c4-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:01:11.586006 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:01:11.585973 5029 generic.go:356] "Generic (PLEG): container finished" podID="0b0c5d07-87e4-4af6-9826-d6b6057e64c4" containerID="533eb6d8a619b6edf32ae49b0548fa56412c8ad8794b0f427f57760c58b411fe" exitCode=0 Oct 30 13:01:11.586468 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:01:11.586024 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww" Oct 30 13:01:11.586468 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:01:11.586036 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww" event={"ID":"0b0c5d07-87e4-4af6-9826-d6b6057e64c4","Type":"ContainerDied","Data":"533eb6d8a619b6edf32ae49b0548fa56412c8ad8794b0f427f57760c58b411fe"} Oct 30 13:01:11.586468 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:01:11.586095 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww" event={"ID":"0b0c5d07-87e4-4af6-9826-d6b6057e64c4","Type":"ContainerDied","Data":"44030882304a11fc48c50de375bdef03072e769402a230c8ac5a25827c2dd549"} Oct 30 13:01:11.586468 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:01:11.586118 5029 scope.go:122] "RemoveContainer" containerID="533eb6d8a619b6edf32ae49b0548fa56412c8ad8794b0f427f57760c58b411fe" Oct 30 13:01:11.594394 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:01:11.594374 5029 scope.go:122] "RemoveContainer" containerID="533eb6d8a619b6edf32ae49b0548fa56412c8ad8794b0f427f57760c58b411fe" Oct 30 13:01:11.594649 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:01:11.594629 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"533eb6d8a619b6edf32ae49b0548fa56412c8ad8794b0f427f57760c58b411fe\": container with ID starting with 533eb6d8a619b6edf32ae49b0548fa56412c8ad8794b0f427f57760c58b411fe not found: ID does not exist" containerID="533eb6d8a619b6edf32ae49b0548fa56412c8ad8794b0f427f57760c58b411fe" Oct 30 13:01:11.594692 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:01:11.594658 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"533eb6d8a619b6edf32ae49b0548fa56412c8ad8794b0f427f57760c58b411fe"} err="failed to get container status \"533eb6d8a619b6edf32ae49b0548fa56412c8ad8794b0f427f57760c58b411fe\": rpc error: code = NotFound desc = could not find container \"533eb6d8a619b6edf32ae49b0548fa56412c8ad8794b0f427f57760c58b411fe\": container with ID starting with 533eb6d8a619b6edf32ae49b0548fa56412c8ad8794b0f427f57760c58b411fe not found: ID does not exist" Oct 30 13:01:11.637792 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:01:11.637760 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww"] Oct 30 13:01:11.648882 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:01:11.648852 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-lf7ww"] Oct 30 13:01:13.088088 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:01:13.088039 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b0c5d07-87e4-4af6-9826-d6b6057e64c4" path="/var/lib/kubelet/pods/0b0c5d07-87e4-4af6-9826-d6b6057e64c4/volumes" Oct 30 13:01:14.878331 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:01:14.878296 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:01:41.136448 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:01:41.136417 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/2.log" Oct 30 13:01:41.138336 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:01:41.137954 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/2.log" Oct 30 13:01:44.878419 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:01:44.878381 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:02:14.877711 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:02:14.877670 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:02:23.695887 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.695851 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr"] Oct 30 13:02:23.696431 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.696250 5029 cpu_manager.go:414] "containerMap: removing container" podUID="0b0c5d07-87e4-4af6-9826-d6b6057e64c4" containerName="manager" Oct 30 13:02:23.696431 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.696264 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b0c5d07-87e4-4af6-9826-d6b6057e64c4" containerName="manager" Oct 30 13:02:23.696431 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.696349 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b0c5d07-87e4-4af6-9826-d6b6057e64c4" containerName="manager" Oct 30 13:02:23.698380 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.698366 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.735241 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.735199 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl"] Oct 30 13:02:23.737782 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.737759 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr"] Oct 30 13:02:23.737919 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.737869 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.754585 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.754557 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl"] Oct 30 13:02:23.785011 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.784983 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.785170 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.785019 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.785170 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.785149 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.785267 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.785192 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.785267 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.785220 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.785267 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.785242 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.785375 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.785329 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.785375 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.785356 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptvf5\" (UniqueName: \"kubernetes.io/projected/3447ed0f-ef06-4a75-8f49-67c043cf08cb-kube-api-access-ptvf5\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.785452 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.785389 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.886804 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.886767 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.886990 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.886815 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.886990 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.886835 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffmxl\" (UniqueName: \"kubernetes.io/projected/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-kube-api-access-ffmxl\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.886990 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.886855 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.886990 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.886913 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptvf5\" (UniqueName: \"kubernetes.io/projected/3447ed0f-ef06-4a75-8f49-67c043cf08cb-kube-api-access-ptvf5\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.887234 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.887006 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.887234 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.887047 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.887234 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.887108 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.887234 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.887140 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.887234 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.887192 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.887234 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.887222 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.887588 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.887275 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.887588 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.887310 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.887588 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.887367 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.887588 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.887373 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.887588 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.887443 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.887588 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.887460 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.887588 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.887483 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.887588 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.887490 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.887588 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.887500 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.887588 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.887520 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.887969 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.887616 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.887969 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.887817 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.889548 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.889526 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.889783 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.889764 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.913651 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.913631 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.913772 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.913724 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptvf5\" (UniqueName: \"kubernetes.io/projected/3447ed0f-ef06-4a75-8f49-67c043cf08cb-kube-api-access-ptvf5\") pod \"maas-default-gateway-openshift-default-5bc884dd87-h4fjr\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:23.989139 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.989049 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.989139 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.989107 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.989139 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.989142 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.989377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.989157 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.989377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.989196 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffmxl\" (UniqueName: \"kubernetes.io/projected/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-kube-api-access-ffmxl\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.989377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.989323 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.989535 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.989395 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.989535 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.989425 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.989535 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.989452 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.989535 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.989474 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.989738 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.989574 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.989808 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.989788 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.989873 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.989802 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.990049 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.990029 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.991248 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.991231 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:23.991700 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:23.991682 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:24.010186 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:24.010162 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:24.018743 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:24.018724 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:24.018881 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:24.018862 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffmxl\" (UniqueName: \"kubernetes.io/projected/1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b-kube-api-access-ffmxl\") pod \"maas-default-gateway-openshift-default-5bc884dd87-ftzzl\" (UID: \"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:24.052967 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:24.052940 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:24.329523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:24.329480 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr"] Oct 30 13:02:24.333831 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:02:24.333803 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3447ed0f_ef06_4a75_8f49_67c043cf08cb.slice/crio-32afcccb6b11ca37b3f621b3973b1e535fb3b17effef110789763b179fdebaf2 WatchSource:0}: Error finding container 32afcccb6b11ca37b3f621b3973b1e535fb3b17effef110789763b179fdebaf2: Status 404 returned error can't find the container with id 32afcccb6b11ca37b3f621b3973b1e535fb3b17effef110789763b179fdebaf2 Oct 30 13:02:24.335804 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:24.335775 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:02:24.335859 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:24.335840 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:02:24.335891 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:24.335869 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:02:24.387713 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:24.387689 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl"] Oct 30 13:02:24.390874 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:02:24.390852 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e1c1b8f_cd7b_44dc_b55a_cbd7c894a38b.slice/crio-d29e0a039f583797023df80021dc1c86fd424223ece292809732e7eb93144e36 WatchSource:0}: Error finding container d29e0a039f583797023df80021dc1c86fd424223ece292809732e7eb93144e36: Status 404 returned error can't find the container with id d29e0a039f583797023df80021dc1c86fd424223ece292809732e7eb93144e36 Oct 30 13:02:24.393037 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:24.393007 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:02:24.393148 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:24.393103 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:02:24.393218 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:24.393158 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:02:24.927301 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:24.927263 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" event={"ID":"3447ed0f-ef06-4a75-8f49-67c043cf08cb","Type":"ContainerStarted","Data":"0d9a68502d58a92011d992a15466590baa8b5b4d0bc9f0348c1effb1af6d1f95"} Oct 30 13:02:24.927682 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:24.927320 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" event={"ID":"3447ed0f-ef06-4a75-8f49-67c043cf08cb","Type":"ContainerStarted","Data":"32afcccb6b11ca37b3f621b3973b1e535fb3b17effef110789763b179fdebaf2"} Oct 30 13:02:24.929484 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:24.929458 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" event={"ID":"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b","Type":"ContainerStarted","Data":"542e9d481e7f72d0f3d40cb27b30758016d307a7221ff1c7858d96eda709d4ee"} Oct 30 13:02:24.929615 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:24.929499 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" event={"ID":"1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b","Type":"ContainerStarted","Data":"d29e0a039f583797023df80021dc1c86fd424223ece292809732e7eb93144e36"} Oct 30 13:02:24.994088 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:24.993881 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" podStartSLOduration=1.993861247 podStartE2EDuration="1.993861247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:02:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:02:24.991786059 +0000 UTC m=+644.017427327" watchObservedRunningTime="2026-10-30 13:02:24.993861247 +0000 UTC m=+644.019502487" Oct 30 13:02:25.016112 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:25.015423 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:25.020087 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:25.019487 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-h4fjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Oct 30 13:02:25.021043 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:25.020785 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" podUID="3447ed0f-ef06-4a75-8f49-67c043cf08cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Oct 30 13:02:25.053228 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:25.053200 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:25.053984 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:25.053935 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" podStartSLOduration=2.053909794 podStartE2EDuration="2.053909794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:02:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:02:25.051827855 +0000 UTC m=+644.077469110" watchObservedRunningTime="2026-10-30 13:02:25.053909794 +0000 UTC m=+644.079551045" Oct 30 13:02:25.054761 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:25.054738 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-ftzzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Oct 30 13:02:25.054864 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:25.054783 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" podUID="1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Oct 30 13:02:26.020110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:26.014495 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-h4fjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Oct 30 13:02:26.020110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:26.014557 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" podUID="3447ed0f-ef06-4a75-8f49-67c043cf08cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Oct 30 13:02:26.053330 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:26.053295 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-ftzzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Oct 30 13:02:26.053493 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:26.053345 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" podUID="1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Oct 30 13:02:27.023610 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:27.023568 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-h4fjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Oct 30 13:02:27.024114 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:27.023623 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" podUID="3447ed0f-ef06-4a75-8f49-67c043cf08cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Oct 30 13:02:27.053384 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:27.053357 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-ftzzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Oct 30 13:02:27.053511 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:27.053402 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" podUID="1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Oct 30 13:02:28.018524 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:28.018483 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-h4fjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Oct 30 13:02:28.018835 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:28.018806 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" podUID="3447ed0f-ef06-4a75-8f49-67c043cf08cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Oct 30 13:02:28.054158 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:28.054115 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-ftzzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Oct 30 13:02:28.054523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:28.054175 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" podUID="1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Oct 30 13:02:29.012091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:29.011456 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-h4fjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Oct 30 13:02:29.012091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:29.011518 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" podUID="3447ed0f-ef06-4a75-8f49-67c043cf08cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Oct 30 13:02:29.054495 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:29.054463 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-ftzzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Oct 30 13:02:29.054875 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:29.054526 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" podUID="1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Oct 30 13:02:30.019090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:30.018462 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-h4fjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Oct 30 13:02:30.019090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:30.018525 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" podUID="3447ed0f-ef06-4a75-8f49-67c043cf08cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Oct 30 13:02:30.054245 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:30.054213 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-ftzzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Oct 30 13:02:30.054392 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:30.054264 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" podUID="1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Oct 30 13:02:31.011154 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:31.011120 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-h4fjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Oct 30 13:02:31.011533 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:31.011170 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" podUID="3447ed0f-ef06-4a75-8f49-67c043cf08cb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Oct 30 13:02:31.053971 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:31.053937 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-ftzzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Oct 30 13:02:31.054161 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:31.053997 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" podUID="1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Oct 30 13:02:32.598167 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:32.598122 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:32.598984 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:32.598961 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:32.599735 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:32.599710 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:02:32.743937 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:32.743903 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:32.744671 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:32.744614 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:32.745436 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:32.745410 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" Oct 30 13:02:44.878305 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:02:44.878268 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:02:54.536308 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:54.536226 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp"] Oct 30 13:02:54.539691 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:54.539659 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" Oct 30 13:02:54.565896 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:54.565872 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-3048f9-serving-cert\"" Oct 30 13:02:54.568751 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:54.568725 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp"] Oct 30 13:02:54.598811 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:54.598783 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7717496d-5ae0-421f-80c8-b89810a6b1c6-metrics-certs\") pod \"maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp\" (UID: \"7717496d-5ae0-421f-80c8-b89810a6b1c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" Oct 30 13:02:54.598939 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:54.598822 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7717496d-5ae0-421f-80c8-b89810a6b1c6-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp\" (UID: \"7717496d-5ae0-421f-80c8-b89810a6b1c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" Oct 30 13:02:54.598939 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:54.598843 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs5nf\" (UniqueName: \"kubernetes.io/projected/7717496d-5ae0-421f-80c8-b89810a6b1c6-kube-api-access-bs5nf\") pod \"maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp\" (UID: \"7717496d-5ae0-421f-80c8-b89810a6b1c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" Oct 30 13:02:54.599022 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:54.599000 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7717496d-5ae0-421f-80c8-b89810a6b1c6-maas-api-tls\") pod \"maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp\" (UID: \"7717496d-5ae0-421f-80c8-b89810a6b1c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" Oct 30 13:02:54.699772 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:54.699744 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7717496d-5ae0-421f-80c8-b89810a6b1c6-maas-api-tls\") pod \"maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp\" (UID: \"7717496d-5ae0-421f-80c8-b89810a6b1c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" Oct 30 13:02:54.699963 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:54.699808 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7717496d-5ae0-421f-80c8-b89810a6b1c6-metrics-certs\") pod \"maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp\" (UID: \"7717496d-5ae0-421f-80c8-b89810a6b1c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" Oct 30 13:02:54.699963 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:54.699845 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7717496d-5ae0-421f-80c8-b89810a6b1c6-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp\" (UID: \"7717496d-5ae0-421f-80c8-b89810a6b1c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" Oct 30 13:02:54.699963 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:54.699863 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs5nf\" (UniqueName: \"kubernetes.io/projected/7717496d-5ae0-421f-80c8-b89810a6b1c6-kube-api-access-bs5nf\") pod \"maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp\" (UID: \"7717496d-5ae0-421f-80c8-b89810a6b1c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" Oct 30 13:02:54.699963 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:02:54.699903 5029 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-serving-cert: secret "maas-api-e2e-worker-w3-3048f9-serving-cert" not found Oct 30 13:02:54.700305 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:02:54.699978 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7717496d-5ae0-421f-80c8-b89810a6b1c6-maas-api-tls podName:7717496d-5ae0-421f-80c8-b89810a6b1c6 nodeName:}" failed. No retries permitted until 2026-10-30 13:02:55.199957185 +0000 UTC m=+674.225598428 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7717496d-5ae0-421f-80c8-b89810a6b1c6-maas-api-tls") pod "maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" (UID: "7717496d-5ae0-421f-80c8-b89810a6b1c6") : secret "maas-api-e2e-worker-w3-3048f9-serving-cert" not found Oct 30 13:02:54.700688 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:54.700668 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7717496d-5ae0-421f-80c8-b89810a6b1c6-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp\" (UID: \"7717496d-5ae0-421f-80c8-b89810a6b1c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" Oct 30 13:02:54.702632 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:54.702613 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7717496d-5ae0-421f-80c8-b89810a6b1c6-metrics-certs\") pod \"maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp\" (UID: \"7717496d-5ae0-421f-80c8-b89810a6b1c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" Oct 30 13:02:54.723515 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:54.723494 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs5nf\" (UniqueName: \"kubernetes.io/projected/7717496d-5ae0-421f-80c8-b89810a6b1c6-kube-api-access-bs5nf\") pod \"maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp\" (UID: \"7717496d-5ae0-421f-80c8-b89810a6b1c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" Oct 30 13:02:55.204587 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:55.204533 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7717496d-5ae0-421f-80c8-b89810a6b1c6-maas-api-tls\") pod \"maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp\" (UID: \"7717496d-5ae0-421f-80c8-b89810a6b1c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" Oct 30 13:02:55.207637 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:55.207606 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7717496d-5ae0-421f-80c8-b89810a6b1c6-maas-api-tls\") pod \"maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp\" (UID: \"7717496d-5ae0-421f-80c8-b89810a6b1c6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" Oct 30 13:02:55.451058 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:55.451026 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" Oct 30 13:02:55.748787 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:55.748691 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp"] Oct 30 13:02:55.753241 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:02:55.753210 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7717496d_5ae0_421f_80c8_b89810a6b1c6.slice/crio-e3757ef9fa760de2cf3b995d166a49f5c56c7514f5651a3694d691c61362e8ee WatchSource:0}: Error finding container e3757ef9fa760de2cf3b995d166a49f5c56c7514f5651a3694d691c61362e8ee: Status 404 returned error can't find the container with id e3757ef9fa760de2cf3b995d166a49f5c56c7514f5651a3694d691c61362e8ee Oct 30 13:02:55.754598 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:55.754579 5029 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 13:02:56.072485 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:56.072451 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" event={"ID":"7717496d-5ae0-421f-80c8-b89810a6b1c6","Type":"ContainerStarted","Data":"e3757ef9fa760de2cf3b995d166a49f5c56c7514f5651a3694d691c61362e8ee"} Oct 30 13:02:56.602895 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:56.602865 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b"] Oct 30 13:02:56.605428 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:56.605410 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" Oct 30 13:02:56.621397 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:56.621370 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-bb71e2-serving-cert\"" Oct 30 13:02:56.635445 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:56.635413 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b"] Oct 30 13:02:56.717706 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:56.717674 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b\" (UID: \"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" Oct 30 13:02:56.717880 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:56.717712 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-metrics-certs\") pod \"maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b\" (UID: \"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" Oct 30 13:02:56.717880 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:56.717732 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-maas-api-tls\") pod \"maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b\" (UID: \"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" Oct 30 13:02:56.717880 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:56.717858 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s8rr\" (UniqueName: \"kubernetes.io/projected/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-kube-api-access-4s8rr\") pod \"maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b\" (UID: \"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" Oct 30 13:02:56.818848 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:56.818816 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b\" (UID: \"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" Oct 30 13:02:56.819338 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:56.818856 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-metrics-certs\") pod \"maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b\" (UID: \"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" Oct 30 13:02:56.819338 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:56.818874 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-maas-api-tls\") pod \"maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b\" (UID: \"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" Oct 30 13:02:56.819338 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:56.818908 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s8rr\" (UniqueName: \"kubernetes.io/projected/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-kube-api-access-4s8rr\") pod \"maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b\" (UID: \"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" Oct 30 13:02:56.819654 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:56.819632 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b\" (UID: \"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" Oct 30 13:02:56.821230 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:56.821211 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-metrics-certs\") pod \"maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b\" (UID: \"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" Oct 30 13:02:56.821315 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:56.821254 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-maas-api-tls\") pod \"maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b\" (UID: \"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" Oct 30 13:02:56.845138 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:56.845104 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s8rr\" (UniqueName: \"kubernetes.io/projected/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-kube-api-access-4s8rr\") pod \"maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b\" (UID: \"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" Oct 30 13:02:56.917304 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:56.917234 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" Oct 30 13:02:57.078715 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:57.078679 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" event={"ID":"7717496d-5ae0-421f-80c8-b89810a6b1c6","Type":"ContainerStarted","Data":"94794eac749679d680fc2ed8f63e4ba43abc520fb76c6913e9b6ade33525c650"} Oct 30 13:02:57.078882 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:57.078801 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" Oct 30 13:02:57.121419 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:57.121372 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" podStartSLOduration=2.661733035 podStartE2EDuration="3.121358206s" totalImagesPullingTime="459.625171ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:02:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:02:57.11993734 +0000 UTC m=+676.145578588" watchObservedRunningTime="2026-10-30 13:02:57.121358206 +0000 UTC m=+676.146999453" Oct 30 13:02:57.225590 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:57.225479 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b"] Oct 30 13:02:57.229985 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:02:57.229958 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60a01580_b95c_44c9_ae4a_8e0c7d3cb2c4.slice/crio-890dfd7d85bb884f9413a38e90180fbb9bd57edbe7915e417874bd2f0c3a593e WatchSource:0}: Error finding container 890dfd7d85bb884f9413a38e90180fbb9bd57edbe7915e417874bd2f0c3a593e: Status 404 returned error can't find the container with id 890dfd7d85bb884f9413a38e90180fbb9bd57edbe7915e417874bd2f0c3a593e Oct 30 13:02:58.085980 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:58.085944 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" event={"ID":"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4","Type":"ContainerStarted","Data":"eb71e1d2b394cdb4145fe1893c8bc352f5aa2f7c6995c92f65b662bfc0ee0bbb"} Oct 30 13:02:58.086426 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:58.086000 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" event={"ID":"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4","Type":"ContainerStarted","Data":"890dfd7d85bb884f9413a38e90180fbb9bd57edbe7915e417874bd2f0c3a593e"} Oct 30 13:02:58.086426 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:58.086219 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" Oct 30 13:02:58.128685 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:02:58.128635 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" podStartSLOduration=1.810876325 podStartE2EDuration="2.128620848s" totalImagesPullingTime="317.744523ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:02:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:02:58.126654654 +0000 UTC m=+677.152295922" watchObservedRunningTime="2026-10-30 13:02:58.128620848 +0000 UTC m=+677.154262095" Oct 30 13:03:00.383142 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:00.383099 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x"] Oct 30 13:03:00.385793 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:00.385774 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" Oct 30 13:03:00.408579 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:00.408550 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-484607-serving-cert\"" Oct 30 13:03:00.420968 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:00.420943 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x"] Oct 30 13:03:00.554531 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:00.554486 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-maas-api-tls\") pod \"maas-api-e2e-worker-w6-484607-55747bf477-7pb4x\" (UID: \"5f73d872-eaf3-4ea7-977e-51d9ff646b7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" Oct 30 13:03:00.554704 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:00.554539 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-484607-55747bf477-7pb4x\" (UID: \"5f73d872-eaf3-4ea7-977e-51d9ff646b7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" Oct 30 13:03:00.554704 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:00.554643 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-metrics-certs\") pod \"maas-api-e2e-worker-w6-484607-55747bf477-7pb4x\" (UID: \"5f73d872-eaf3-4ea7-977e-51d9ff646b7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" Oct 30 13:03:00.554704 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:00.554676 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf2cp\" (UniqueName: \"kubernetes.io/projected/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-kube-api-access-jf2cp\") pod \"maas-api-e2e-worker-w6-484607-55747bf477-7pb4x\" (UID: \"5f73d872-eaf3-4ea7-977e-51d9ff646b7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" Oct 30 13:03:00.655457 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:00.655353 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-metrics-certs\") pod \"maas-api-e2e-worker-w6-484607-55747bf477-7pb4x\" (UID: \"5f73d872-eaf3-4ea7-977e-51d9ff646b7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" Oct 30 13:03:00.655457 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:00.655407 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf2cp\" (UniqueName: \"kubernetes.io/projected/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-kube-api-access-jf2cp\") pod \"maas-api-e2e-worker-w6-484607-55747bf477-7pb4x\" (UID: \"5f73d872-eaf3-4ea7-977e-51d9ff646b7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" Oct 30 13:03:00.655457 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:00.655463 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-maas-api-tls\") pod \"maas-api-e2e-worker-w6-484607-55747bf477-7pb4x\" (UID: \"5f73d872-eaf3-4ea7-977e-51d9ff646b7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" Oct 30 13:03:00.655727 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:00.655590 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-484607-55747bf477-7pb4x\" (UID: \"5f73d872-eaf3-4ea7-977e-51d9ff646b7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" Oct 30 13:03:00.656273 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:00.656245 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-484607-55747bf477-7pb4x\" (UID: \"5f73d872-eaf3-4ea7-977e-51d9ff646b7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" Oct 30 13:03:00.657797 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:00.657771 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-metrics-certs\") pod \"maas-api-e2e-worker-w6-484607-55747bf477-7pb4x\" (UID: \"5f73d872-eaf3-4ea7-977e-51d9ff646b7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" Oct 30 13:03:00.657968 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:00.657948 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-maas-api-tls\") pod \"maas-api-e2e-worker-w6-484607-55747bf477-7pb4x\" (UID: \"5f73d872-eaf3-4ea7-977e-51d9ff646b7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" Oct 30 13:03:00.684096 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:00.684032 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf2cp\" (UniqueName: \"kubernetes.io/projected/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-kube-api-access-jf2cp\") pod \"maas-api-e2e-worker-w6-484607-55747bf477-7pb4x\" (UID: \"5f73d872-eaf3-4ea7-977e-51d9ff646b7e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" Oct 30 13:03:00.698829 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:00.698800 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" Oct 30 13:03:00.997962 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:00.997867 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x"] Oct 30 13:03:01.001966 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:03:01.001936 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f73d872_eaf3_4ea7_977e_51d9ff646b7e.slice/crio-adcf12613a3db5fea75fbc56039f48fc523e24d2e42b934b8980cab1b7caf9ee WatchSource:0}: Error finding container adcf12613a3db5fea75fbc56039f48fc523e24d2e42b934b8980cab1b7caf9ee: Status 404 returned error can't find the container with id adcf12613a3db5fea75fbc56039f48fc523e24d2e42b934b8980cab1b7caf9ee Oct 30 13:03:01.100276 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:01.100242 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" event={"ID":"5f73d872-eaf3-4ea7-977e-51d9ff646b7e","Type":"ContainerStarted","Data":"adcf12613a3db5fea75fbc56039f48fc523e24d2e42b934b8980cab1b7caf9ee"} Oct 30 13:03:02.106815 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:02.106781 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" event={"ID":"5f73d872-eaf3-4ea7-977e-51d9ff646b7e","Type":"ContainerStarted","Data":"2b373c88f1d76ab0d023400ee64b036ff512ed8aabc3d611261ed4d2c86a6856"} Oct 30 13:03:02.107208 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:02.106885 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" Oct 30 13:03:02.148664 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:02.148623 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" podStartSLOduration=1.794882881 podStartE2EDuration="2.148608239s" totalImagesPullingTime="353.725358ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:03:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:03:02.148328901 +0000 UTC m=+681.173970177" watchObservedRunningTime="2026-10-30 13:03:02.148608239 +0000 UTC m=+681.174249489" Oct 30 13:03:03.092687 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:03.092656 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" Oct 30 13:03:04.096458 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:04.096427 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" Oct 30 13:03:04.371788 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:04.371709 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5"] Oct 30 13:03:04.374491 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:04.374469 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" Oct 30 13:03:04.391767 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:04.391747 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-54994c-serving-cert\"" Oct 30 13:03:04.399036 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:04.399008 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5"] Oct 30 13:03:04.495549 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:04.495511 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e23ddba-6739-4ccb-803a-d19cb9501b95-metrics-certs\") pod \"maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5\" (UID: \"0e23ddba-6739-4ccb-803a-d19cb9501b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" Oct 30 13:03:04.495692 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:04.495558 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tcdq\" (UniqueName: \"kubernetes.io/projected/0e23ddba-6739-4ccb-803a-d19cb9501b95-kube-api-access-5tcdq\") pod \"maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5\" (UID: \"0e23ddba-6739-4ccb-803a-d19cb9501b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" Oct 30 13:03:04.495692 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:04.495635 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e23ddba-6739-4ccb-803a-d19cb9501b95-maas-api-tls\") pod \"maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5\" (UID: \"0e23ddba-6739-4ccb-803a-d19cb9501b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" Oct 30 13:03:04.495790 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:04.495753 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e23ddba-6739-4ccb-803a-d19cb9501b95-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5\" (UID: \"0e23ddba-6739-4ccb-803a-d19cb9501b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" Oct 30 13:03:04.596427 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:04.596392 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e23ddba-6739-4ccb-803a-d19cb9501b95-metrics-certs\") pod \"maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5\" (UID: \"0e23ddba-6739-4ccb-803a-d19cb9501b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" Oct 30 13:03:04.596606 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:04.596438 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tcdq\" (UniqueName: \"kubernetes.io/projected/0e23ddba-6739-4ccb-803a-d19cb9501b95-kube-api-access-5tcdq\") pod \"maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5\" (UID: \"0e23ddba-6739-4ccb-803a-d19cb9501b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" Oct 30 13:03:04.596606 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:04.596536 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e23ddba-6739-4ccb-803a-d19cb9501b95-maas-api-tls\") pod \"maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5\" (UID: \"0e23ddba-6739-4ccb-803a-d19cb9501b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" Oct 30 13:03:04.596691 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:04.596640 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e23ddba-6739-4ccb-803a-d19cb9501b95-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5\" (UID: \"0e23ddba-6739-4ccb-803a-d19cb9501b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" Oct 30 13:03:04.597286 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:04.597260 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e23ddba-6739-4ccb-803a-d19cb9501b95-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5\" (UID: \"0e23ddba-6739-4ccb-803a-d19cb9501b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" Oct 30 13:03:04.598830 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:04.598803 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e23ddba-6739-4ccb-803a-d19cb9501b95-maas-api-tls\") pod \"maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5\" (UID: \"0e23ddba-6739-4ccb-803a-d19cb9501b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" Oct 30 13:03:04.598926 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:04.598906 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e23ddba-6739-4ccb-803a-d19cb9501b95-metrics-certs\") pod \"maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5\" (UID: \"0e23ddba-6739-4ccb-803a-d19cb9501b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" Oct 30 13:03:04.620296 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:04.620272 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tcdq\" (UniqueName: \"kubernetes.io/projected/0e23ddba-6739-4ccb-803a-d19cb9501b95-kube-api-access-5tcdq\") pod \"maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5\" (UID: \"0e23ddba-6739-4ccb-803a-d19cb9501b95\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" Oct 30 13:03:04.693989 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:04.693909 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" Oct 30 13:03:05.001722 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:05.001638 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5"] Oct 30 13:03:05.005443 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:03:05.005415 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e23ddba_6739_4ccb_803a_d19cb9501b95.slice/crio-bac1bf406f4fe350e833c591bb59a3c702bd1fd82efa50ff21437e14c0308447 WatchSource:0}: Error finding container bac1bf406f4fe350e833c591bb59a3c702bd1fd82efa50ff21437e14c0308447: Status 404 returned error can't find the container with id bac1bf406f4fe350e833c591bb59a3c702bd1fd82efa50ff21437e14c0308447 Oct 30 13:03:05.120932 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:05.120895 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" event={"ID":"0e23ddba-6739-4ccb-803a-d19cb9501b95","Type":"ContainerStarted","Data":"bac1bf406f4fe350e833c591bb59a3c702bd1fd82efa50ff21437e14c0308447"} Oct 30 13:03:06.128033 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:06.127997 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" event={"ID":"0e23ddba-6739-4ccb-803a-d19cb9501b95","Type":"ContainerStarted","Data":"d8d99b32b06183bd0d8eeacce1973675d964d950589552ee722d8bfa30d060bf"} Oct 30 13:03:06.128535 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:06.128194 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" Oct 30 13:03:06.170378 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:06.170329 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" podStartSLOduration=1.774190663 podStartE2EDuration="2.170313225s" totalImagesPullingTime="396.122562ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:03:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:03:06.168854148 +0000 UTC m=+685.194495430" watchObservedRunningTime="2026-10-30 13:03:06.170313225 +0000 UTC m=+685.195954478" Oct 30 13:03:06.208537 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:06.208504 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g"] Oct 30 13:03:06.211296 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:06.211276 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" Oct 30 13:03:06.229395 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:06.229372 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-42ee9d-serving-cert\"" Oct 30 13:03:06.238319 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:06.238261 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g"] Oct 30 13:03:06.312948 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:06.312914 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2080a9e8-70bc-4c2e-8b67-7bb00077837b-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g\" (UID: \"2080a9e8-70bc-4c2e-8b67-7bb00077837b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" Oct 30 13:03:06.313127 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:06.313091 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2080a9e8-70bc-4c2e-8b67-7bb00077837b-maas-api-tls\") pod \"maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g\" (UID: \"2080a9e8-70bc-4c2e-8b67-7bb00077837b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" Oct 30 13:03:06.313240 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:06.313225 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czv5p\" (UniqueName: \"kubernetes.io/projected/2080a9e8-70bc-4c2e-8b67-7bb00077837b-kube-api-access-czv5p\") pod \"maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g\" (UID: \"2080a9e8-70bc-4c2e-8b67-7bb00077837b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" Oct 30 13:03:06.313291 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:06.313260 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2080a9e8-70bc-4c2e-8b67-7bb00077837b-metrics-certs\") pod \"maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g\" (UID: \"2080a9e8-70bc-4c2e-8b67-7bb00077837b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" Oct 30 13:03:06.414121 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:06.414022 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czv5p\" (UniqueName: \"kubernetes.io/projected/2080a9e8-70bc-4c2e-8b67-7bb00077837b-kube-api-access-czv5p\") pod \"maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g\" (UID: \"2080a9e8-70bc-4c2e-8b67-7bb00077837b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" Oct 30 13:03:06.414121 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:06.414091 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2080a9e8-70bc-4c2e-8b67-7bb00077837b-metrics-certs\") pod \"maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g\" (UID: \"2080a9e8-70bc-4c2e-8b67-7bb00077837b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" Oct 30 13:03:06.414303 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:06.414134 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2080a9e8-70bc-4c2e-8b67-7bb00077837b-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g\" (UID: \"2080a9e8-70bc-4c2e-8b67-7bb00077837b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" Oct 30 13:03:06.414303 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:06.414194 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2080a9e8-70bc-4c2e-8b67-7bb00077837b-maas-api-tls\") pod \"maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g\" (UID: \"2080a9e8-70bc-4c2e-8b67-7bb00077837b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" Oct 30 13:03:06.414882 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:06.414853 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2080a9e8-70bc-4c2e-8b67-7bb00077837b-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g\" (UID: \"2080a9e8-70bc-4c2e-8b67-7bb00077837b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" Oct 30 13:03:06.416549 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:06.416531 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2080a9e8-70bc-4c2e-8b67-7bb00077837b-metrics-certs\") pod \"maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g\" (UID: \"2080a9e8-70bc-4c2e-8b67-7bb00077837b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" Oct 30 13:03:06.416655 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:06.416592 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2080a9e8-70bc-4c2e-8b67-7bb00077837b-maas-api-tls\") pod \"maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g\" (UID: \"2080a9e8-70bc-4c2e-8b67-7bb00077837b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" Oct 30 13:03:06.442758 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:06.442723 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czv5p\" (UniqueName: \"kubernetes.io/projected/2080a9e8-70bc-4c2e-8b67-7bb00077837b-kube-api-access-czv5p\") pod \"maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g\" (UID: \"2080a9e8-70bc-4c2e-8b67-7bb00077837b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" Oct 30 13:03:06.523604 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:06.523573 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" Oct 30 13:03:06.822250 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:06.822216 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g"] Oct 30 13:03:06.826311 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:03:06.826284 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2080a9e8_70bc_4c2e_8b67_7bb00077837b.slice/crio-207e6b973337a057da7a0b513468395e00b53d3e87a0aed97cc3bb29384392e3 WatchSource:0}: Error finding container 207e6b973337a057da7a0b513468395e00b53d3e87a0aed97cc3bb29384392e3: Status 404 returned error can't find the container with id 207e6b973337a057da7a0b513468395e00b53d3e87a0aed97cc3bb29384392e3 Oct 30 13:03:07.133861 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:07.133769 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" event={"ID":"2080a9e8-70bc-4c2e-8b67-7bb00077837b","Type":"ContainerStarted","Data":"207e6b973337a057da7a0b513468395e00b53d3e87a0aed97cc3bb29384392e3"} Oct 30 13:03:08.117660 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:08.117631 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" Oct 30 13:03:08.139856 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:08.139818 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" event={"ID":"2080a9e8-70bc-4c2e-8b67-7bb00077837b","Type":"ContainerStarted","Data":"38ddc19bab16e2d6edb67f66cc366e123fed7ee98c03c459e3ce713cdbdfa711"} Oct 30 13:03:08.140346 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:08.139889 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" Oct 30 13:03:08.202632 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:08.202584 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" podStartSLOduration=1.8621676470000001 podStartE2EDuration="2.202568817s" totalImagesPullingTime="340.40117ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:03:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:03:08.201825743 +0000 UTC m=+687.227466991" watchObservedRunningTime="2026-10-30 13:03:08.202568817 +0000 UTC m=+687.228210092" Oct 30 13:03:12.139355 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:12.139325 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" Oct 30 13:03:14.154128 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:14.154084 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" Oct 30 13:03:14.881378 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:03:14.881327 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:03:24.227043 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.227012 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx"] Oct 30 13:03:24.230236 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.230217 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:24.257829 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.257802 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx"] Oct 30 13:03:24.267009 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.266987 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-bb71e2\"/\"simulator-w2-bb71e2-kserve-self-signed-certs\"" Oct 30 13:03:24.267142 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.267045 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-bb71e2\"/\"kube-root-ca.crt\"" Oct 30 13:03:24.267142 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.267057 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-bb71e2\"/\"openshift-service-ca.crt\"" Oct 30 13:03:24.267142 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.267122 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-bb71e2\"/\"default-dockercfg-8lldv\"" Oct 30 13:03:24.284596 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.284566 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-model-cache\") pod \"simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:24.284703 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.284595 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-tmp-dir\") pod \"simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:24.284703 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.284615 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-dshm\") pod \"simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:24.284778 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.284761 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsc2b\" (UniqueName: \"kubernetes.io/projected/d6bdcd3f-0200-48e6-af85-2d1d8f022054-kube-api-access-gsc2b\") pod \"simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:24.284837 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.284825 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-home\") pod \"simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:24.284875 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.284862 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6bdcd3f-0200-48e6-af85-2d1d8f022054-tls-certs\") pod \"simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:24.385707 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.385663 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-home\") pod \"simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:24.385898 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.385720 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6bdcd3f-0200-48e6-af85-2d1d8f022054-tls-certs\") pod \"simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:24.385898 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.385760 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-model-cache\") pod \"simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:24.385898 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.385781 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-tmp-dir\") pod \"simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:24.385898 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.385796 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-dshm\") pod \"simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:24.385898 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.385889 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsc2b\" (UniqueName: \"kubernetes.io/projected/d6bdcd3f-0200-48e6-af85-2d1d8f022054-kube-api-access-gsc2b\") pod \"simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:24.386314 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.386293 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-home\") pod \"simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:24.386392 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.386330 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-tmp-dir\") pod \"simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:24.386392 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.386352 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-model-cache\") pod \"simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:24.388201 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.388180 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-dshm\") pod \"simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:24.388577 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.388560 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6bdcd3f-0200-48e6-af85-2d1d8f022054-tls-certs\") pod \"simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:24.432044 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.432013 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsc2b\" (UniqueName: \"kubernetes.io/projected/d6bdcd3f-0200-48e6-af85-2d1d8f022054-kube-api-access-gsc2b\") pod \"simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:24.541423 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.541097 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:24.855031 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:24.854992 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx"] Oct 30 13:03:25.226842 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:25.226723 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" event={"ID":"d6bdcd3f-0200-48e6-af85-2d1d8f022054","Type":"ContainerStarted","Data":"613594ad4a961182d891fb06206dd1aa7aa3c0ecfc11ffe6a7b2af31da0174d2"} Oct 30 13:03:26.232595 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:26.232510 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" event={"ID":"d6bdcd3f-0200-48e6-af85-2d1d8f022054","Type":"ContainerStarted","Data":"0bd86e413d69efb0fb7226ec133d467fc74145d8b04c512bba2c836baabcca00"} Oct 30 13:03:26.276667 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:26.276618 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" podStartSLOduration=1.2387631510000001 podStartE2EDuration="2.276605691s" totalImagesPullingTime="1.03784254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:03:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:03:26.27489242 +0000 UTC m=+705.300533691" watchObservedRunningTime="2026-10-30 13:03:26.276605691 +0000 UTC m=+705.302246940" Oct 30 13:03:26.541673 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:26.541594 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:32.708262 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.708229 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p"] Oct 30 13:03:32.713574 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.713550 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:32.734901 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.734863 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p"] Oct 30 13:03:32.743128 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.743083 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-175831\"/\"openshift-service-ca.crt\"" Oct 30 13:03:32.743284 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.743126 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-175831\"/\"simulator-w5-175831-kserve-self-signed-certs\"" Oct 30 13:03:32.743284 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.743161 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-175831\"/\"kube-root-ca.crt\"" Oct 30 13:03:32.743284 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.743217 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-175831\"/\"default-dockercfg-png5h\"" Oct 30 13:03:32.765986 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.765959 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-tmp-dir\") pod \"simulator-w5-175831-kserve-5b7646f447-7p64p\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:32.766182 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.766164 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-model-cache\") pod \"simulator-w5-175831-kserve-5b7646f447-7p64p\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:32.766259 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.766217 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-home\") pod \"simulator-w5-175831-kserve-5b7646f447-7p64p\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:32.766312 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.766292 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-dshm\") pod \"simulator-w5-175831-kserve-5b7646f447-7p64p\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:32.766349 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.766332 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19409b77-526f-48f7-b539-6d8ebb2d3dfc-tls-certs\") pod \"simulator-w5-175831-kserve-5b7646f447-7p64p\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:32.766388 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.766349 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4mvr\" (UniqueName: \"kubernetes.io/projected/19409b77-526f-48f7-b539-6d8ebb2d3dfc-kube-api-access-n4mvr\") pod \"simulator-w5-175831-kserve-5b7646f447-7p64p\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:32.867247 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.867216 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-model-cache\") pod \"simulator-w5-175831-kserve-5b7646f447-7p64p\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:32.867438 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.867262 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-home\") pod \"simulator-w5-175831-kserve-5b7646f447-7p64p\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:32.867438 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.867294 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-dshm\") pod \"simulator-w5-175831-kserve-5b7646f447-7p64p\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:32.867438 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.867329 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19409b77-526f-48f7-b539-6d8ebb2d3dfc-tls-certs\") pod \"simulator-w5-175831-kserve-5b7646f447-7p64p\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:32.867438 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.867352 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4mvr\" (UniqueName: \"kubernetes.io/projected/19409b77-526f-48f7-b539-6d8ebb2d3dfc-kube-api-access-n4mvr\") pod \"simulator-w5-175831-kserve-5b7646f447-7p64p\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:32.867438 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.867407 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-tmp-dir\") pod \"simulator-w5-175831-kserve-5b7646f447-7p64p\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:32.867756 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.867729 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-model-cache\") pod \"simulator-w5-175831-kserve-5b7646f447-7p64p\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:32.867828 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.867755 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-home\") pod \"simulator-w5-175831-kserve-5b7646f447-7p64p\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:32.867828 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.867789 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-tmp-dir\") pod \"simulator-w5-175831-kserve-5b7646f447-7p64p\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:32.869820 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.869799 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-dshm\") pod \"simulator-w5-175831-kserve-5b7646f447-7p64p\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:32.870101 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.870079 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19409b77-526f-48f7-b539-6d8ebb2d3dfc-tls-certs\") pod \"simulator-w5-175831-kserve-5b7646f447-7p64p\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:32.891048 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:32.891026 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4mvr\" (UniqueName: \"kubernetes.io/projected/19409b77-526f-48f7-b539-6d8ebb2d3dfc-kube-api-access-n4mvr\") pod \"simulator-w5-175831-kserve-5b7646f447-7p64p\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:33.025322 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:33.025234 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:33.357431 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:33.357397 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p"] Oct 30 13:03:33.361505 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:03:33.361480 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19409b77_526f_48f7_b539_6d8ebb2d3dfc.slice/crio-73acb5a510cf98ef6502b8130955b575216fc4598965fd60ee62d18d41700b55 WatchSource:0}: Error finding container 73acb5a510cf98ef6502b8130955b575216fc4598965fd60ee62d18d41700b55: Status 404 returned error can't find the container with id 73acb5a510cf98ef6502b8130955b575216fc4598965fd60ee62d18d41700b55 Oct 30 13:03:34.275084 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:34.275028 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" event={"ID":"19409b77-526f-48f7-b539-6d8ebb2d3dfc","Type":"ContainerStarted","Data":"3dd6aae5c749df07ad715462c10bdccc1235f5a7fc30caf34e834c7df061bf06"} Oct 30 13:03:34.275508 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:34.275091 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" event={"ID":"19409b77-526f-48f7-b539-6d8ebb2d3dfc","Type":"ContainerStarted","Data":"73acb5a510cf98ef6502b8130955b575216fc4598965fd60ee62d18d41700b55"} Oct 30 13:03:34.328429 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:34.328378 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" podStartSLOduration=2.328363474 podStartE2EDuration="2.328363474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:03:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:03:34.327486796 +0000 UTC m=+713.353128062" watchObservedRunningTime="2026-10-30 13:03:34.328363474 +0000 UTC m=+713.354004723" Oct 30 13:03:34.541731 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:34.541653 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:34.554847 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:34.554816 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:35.026335 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:35.026304 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:35.296510 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:35.296420 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:03:39.667196 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.667158 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb"] Oct 30 13:03:39.671874 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.671850 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:39.692698 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.692669 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb"] Oct 30 13:03:39.701179 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.701157 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-42ee9d\"/\"default-dockercfg-dn7vg\"" Oct 30 13:03:39.701359 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.701250 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-42ee9d\"/\"kube-root-ca.crt\"" Oct 30 13:03:39.701359 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.701285 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-42ee9d\"/\"simulator-w0-42ee9d-kserve-self-signed-certs\"" Oct 30 13:03:39.701503 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.701314 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-42ee9d\"/\"openshift-service-ca.crt\"" Oct 30 13:03:39.836804 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.836761 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5jl5\" (UniqueName: \"kubernetes.io/projected/99e695e7-e73c-429c-87ca-39ee740917c1-kube-api-access-n5jl5\") pod \"simulator-w0-42ee9d-kserve-575877dfdd-rdcnb\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:39.836804 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.836809 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-tmp-dir\") pod \"simulator-w0-42ee9d-kserve-575877dfdd-rdcnb\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:39.837009 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.836832 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99e695e7-e73c-429c-87ca-39ee740917c1-tls-certs\") pod \"simulator-w0-42ee9d-kserve-575877dfdd-rdcnb\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:39.837009 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.836910 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-home\") pod \"simulator-w0-42ee9d-kserve-575877dfdd-rdcnb\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:39.837009 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.836947 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-model-cache\") pod \"simulator-w0-42ee9d-kserve-575877dfdd-rdcnb\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:39.837128 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.837016 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-dshm\") pod \"simulator-w0-42ee9d-kserve-575877dfdd-rdcnb\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:39.938488 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.938379 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5jl5\" (UniqueName: \"kubernetes.io/projected/99e695e7-e73c-429c-87ca-39ee740917c1-kube-api-access-n5jl5\") pod \"simulator-w0-42ee9d-kserve-575877dfdd-rdcnb\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:39.938488 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.938430 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-tmp-dir\") pod \"simulator-w0-42ee9d-kserve-575877dfdd-rdcnb\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:39.938488 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.938458 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99e695e7-e73c-429c-87ca-39ee740917c1-tls-certs\") pod \"simulator-w0-42ee9d-kserve-575877dfdd-rdcnb\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:39.938797 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.938489 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-home\") pod \"simulator-w0-42ee9d-kserve-575877dfdd-rdcnb\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:39.938797 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.938515 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-model-cache\") pod \"simulator-w0-42ee9d-kserve-575877dfdd-rdcnb\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:39.938797 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.938557 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-dshm\") pod \"simulator-w0-42ee9d-kserve-575877dfdd-rdcnb\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:39.938941 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.938912 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-tmp-dir\") pod \"simulator-w0-42ee9d-kserve-575877dfdd-rdcnb\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:39.939138 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.939044 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-home\") pod \"simulator-w0-42ee9d-kserve-575877dfdd-rdcnb\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:39.939138 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.939111 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-model-cache\") pod \"simulator-w0-42ee9d-kserve-575877dfdd-rdcnb\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:39.941734 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.941705 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-dshm\") pod \"simulator-w0-42ee9d-kserve-575877dfdd-rdcnb\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:39.941913 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.941895 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99e695e7-e73c-429c-87ca-39ee740917c1-tls-certs\") pod \"simulator-w0-42ee9d-kserve-575877dfdd-rdcnb\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:39.962444 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.962421 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5jl5\" (UniqueName: \"kubernetes.io/projected/99e695e7-e73c-429c-87ca-39ee740917c1-kube-api-access-n5jl5\") pod \"simulator-w0-42ee9d-kserve-575877dfdd-rdcnb\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:39.985437 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:39.985414 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:40.301238 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:40.300985 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb"] Oct 30 13:03:40.304764 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:03:40.304735 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99e695e7_e73c_429c_87ca_39ee740917c1.slice/crio-d44a3e56f225a374066d534932daa5a50855dd79518453f91316dcb0cbb36e33 WatchSource:0}: Error finding container d44a3e56f225a374066d534932daa5a50855dd79518453f91316dcb0cbb36e33: Status 404 returned error can't find the container with id d44a3e56f225a374066d534932daa5a50855dd79518453f91316dcb0cbb36e33 Oct 30 13:03:41.311553 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:41.311513 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" event={"ID":"99e695e7-e73c-429c-87ca-39ee740917c1","Type":"ContainerStarted","Data":"af2ad9aebf66050565eaea52da7b746a5815002ba3920d9e83536c9f5eb3f236"} Oct 30 13:03:41.311968 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:41.311563 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" event={"ID":"99e695e7-e73c-429c-87ca-39ee740917c1","Type":"ContainerStarted","Data":"d44a3e56f225a374066d534932daa5a50855dd79518453f91316dcb0cbb36e33"} Oct 30 13:03:41.353385 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:41.353335 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" podStartSLOduration=2.353318186 podStartE2EDuration="2.353318186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:03:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:03:41.352942564 +0000 UTC m=+720.378583824" watchObservedRunningTime="2026-10-30 13:03:41.353318186 +0000 UTC m=+720.378959434" Oct 30 13:03:41.985546 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:41.985507 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:43.026149 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:43.026114 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:43.038872 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:43.038844 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:43.332592 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:43.332562 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:03:44.878608 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:03:44.878571 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:03:49.985543 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:49.985504 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:49.997562 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:49.997540 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:50.367969 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:50.367943 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:03:52.175102 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.175053 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44"] Oct 30 13:03:52.179852 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.179822 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:03:52.199346 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.199324 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44"] Oct 30 13:03:52.208686 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.208662 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-ac0c58\"/\"openshift-service-ca.crt\"" Oct 30 13:03:52.208795 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.208686 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-ac0c58\"/\"kube-root-ca.crt\"" Oct 30 13:03:52.208795 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.208693 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-ac0c58\"/\"default-dockercfg-k5q4c\"" Oct 30 13:03:52.208795 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.208778 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-ac0c58\"/\"simulator-w1-ac0c58-kserve-self-signed-certs\"" Oct 30 13:03:52.267413 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.267376 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jrhw\" (UniqueName: \"kubernetes.io/projected/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-kube-api-access-8jrhw\") pod \"simulator-w1-ac0c58-kserve-74b44db96d-rrz44\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:03:52.267570 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.267417 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-model-cache\") pod \"simulator-w1-ac0c58-kserve-74b44db96d-rrz44\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:03:52.267570 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.267492 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-home\") pod \"simulator-w1-ac0c58-kserve-74b44db96d-rrz44\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:03:52.267650 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.267619 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-tmp-dir\") pod \"simulator-w1-ac0c58-kserve-74b44db96d-rrz44\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:03:52.267650 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.267642 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-tls-certs\") pod \"simulator-w1-ac0c58-kserve-74b44db96d-rrz44\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:03:52.267728 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.267673 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-dshm\") pod \"simulator-w1-ac0c58-kserve-74b44db96d-rrz44\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:03:52.369148 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.369116 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-tmp-dir\") pod \"simulator-w1-ac0c58-kserve-74b44db96d-rrz44\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:03:52.369148 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.369151 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-tls-certs\") pod \"simulator-w1-ac0c58-kserve-74b44db96d-rrz44\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:03:52.369362 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.369192 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-dshm\") pod \"simulator-w1-ac0c58-kserve-74b44db96d-rrz44\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:03:52.369362 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.369265 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jrhw\" (UniqueName: \"kubernetes.io/projected/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-kube-api-access-8jrhw\") pod \"simulator-w1-ac0c58-kserve-74b44db96d-rrz44\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:03:52.369436 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.369398 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-model-cache\") pod \"simulator-w1-ac0c58-kserve-74b44db96d-rrz44\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:03:52.369477 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.369446 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-home\") pod \"simulator-w1-ac0c58-kserve-74b44db96d-rrz44\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:03:52.369553 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.369533 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-tmp-dir\") pod \"simulator-w1-ac0c58-kserve-74b44db96d-rrz44\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:03:52.369850 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.369819 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-home\") pod \"simulator-w1-ac0c58-kserve-74b44db96d-rrz44\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:03:52.369850 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.369831 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-model-cache\") pod \"simulator-w1-ac0c58-kserve-74b44db96d-rrz44\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:03:52.371822 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.371790 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-dshm\") pod \"simulator-w1-ac0c58-kserve-74b44db96d-rrz44\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:03:52.371971 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.371955 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-tls-certs\") pod \"simulator-w1-ac0c58-kserve-74b44db96d-rrz44\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:03:52.401417 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.401387 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jrhw\" (UniqueName: \"kubernetes.io/projected/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-kube-api-access-8jrhw\") pod \"simulator-w1-ac0c58-kserve-74b44db96d-rrz44\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:03:52.491406 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.491324 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:03:52.795392 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:52.795311 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44"] Oct 30 13:03:52.799633 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:03:52.799594 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd68fdee_30c0_42ac_bf6e_aff5d2abb46d.slice/crio-406bdfdffb1172cf9fb633674b9c5e1a74d8430a21c3a88d084dcfdb50cba135 WatchSource:0}: Error finding container 406bdfdffb1172cf9fb633674b9c5e1a74d8430a21c3a88d084dcfdb50cba135: Status 404 returned error can't find the container with id 406bdfdffb1172cf9fb633674b9c5e1a74d8430a21c3a88d084dcfdb50cba135 Oct 30 13:03:53.372984 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:53.372951 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" event={"ID":"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d","Type":"ContainerStarted","Data":"00749907ad1885b7e1665dff6f3ad94d2dd244caa067accdd9854339d7030b08"} Oct 30 13:03:53.373417 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:53.372995 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" event={"ID":"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d","Type":"ContainerStarted","Data":"406bdfdffb1172cf9fb633674b9c5e1a74d8430a21c3a88d084dcfdb50cba135"} Oct 30 13:03:53.416179 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:53.416137 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" podStartSLOduration=1.41612261 podStartE2EDuration="1.41612261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:03:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:03:53.414652014 +0000 UTC m=+732.440293283" watchObservedRunningTime="2026-10-30 13:03:53.41612261 +0000 UTC m=+732.441763879" Oct 30 13:03:53.491433 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:03:53.491398 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:04:02.492470 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:02.492439 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:04:02.506011 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:02.505984 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:04:03.436870 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:03.436837 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:04:10.101981 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.101934 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q"] Oct 30 13:04:10.106261 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.106238 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:10.125408 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.125382 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q"] Oct 30 13:04:10.134141 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.134115 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-484607\"/\"openshift-service-ca.crt\"" Oct 30 13:04:10.134282 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.134160 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-484607\"/\"kube-root-ca.crt\"" Oct 30 13:04:10.134282 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.134223 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-484607\"/\"default-dockercfg-v4djb\"" Oct 30 13:04:10.134400 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.134337 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-484607\"/\"premium-w6-484607-kserve-self-signed-certs\"" Oct 30 13:04:10.245199 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.245160 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-dshm\") pod \"premium-w6-484607-kserve-7f8d6b7d8f-fl54q\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:10.245389 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.245214 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6962e138-faa9-438c-9138-1133bdce1011-tls-certs\") pod \"premium-w6-484607-kserve-7f8d6b7d8f-fl54q\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:10.245466 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.245405 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-model-cache\") pod \"premium-w6-484607-kserve-7f8d6b7d8f-fl54q\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:10.245466 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.245436 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-home\") pod \"premium-w6-484607-kserve-7f8d6b7d8f-fl54q\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:10.245542 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.245470 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-tmp-dir\") pod \"premium-w6-484607-kserve-7f8d6b7d8f-fl54q\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:10.245666 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.245631 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4hlt\" (UniqueName: \"kubernetes.io/projected/6962e138-faa9-438c-9138-1133bdce1011-kube-api-access-x4hlt\") pod \"premium-w6-484607-kserve-7f8d6b7d8f-fl54q\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:10.346674 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.346634 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6962e138-faa9-438c-9138-1133bdce1011-tls-certs\") pod \"premium-w6-484607-kserve-7f8d6b7d8f-fl54q\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:10.346868 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.346734 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-model-cache\") pod \"premium-w6-484607-kserve-7f8d6b7d8f-fl54q\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:10.346868 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.346759 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-home\") pod \"premium-w6-484607-kserve-7f8d6b7d8f-fl54q\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:10.346868 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.346785 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-tmp-dir\") pod \"premium-w6-484607-kserve-7f8d6b7d8f-fl54q\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:10.347038 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.346967 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4hlt\" (UniqueName: \"kubernetes.io/projected/6962e138-faa9-438c-9138-1133bdce1011-kube-api-access-x4hlt\") pod \"premium-w6-484607-kserve-7f8d6b7d8f-fl54q\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:10.347329 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.347304 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-home\") pod \"premium-w6-484607-kserve-7f8d6b7d8f-fl54q\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:10.347430 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.347315 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-model-cache\") pod \"premium-w6-484607-kserve-7f8d6b7d8f-fl54q\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:10.347430 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.347412 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-dshm\") pod \"premium-w6-484607-kserve-7f8d6b7d8f-fl54q\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:10.347555 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.347539 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-tmp-dir\") pod \"premium-w6-484607-kserve-7f8d6b7d8f-fl54q\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:10.349456 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.349433 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6962e138-faa9-438c-9138-1133bdce1011-tls-certs\") pod \"premium-w6-484607-kserve-7f8d6b7d8f-fl54q\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:10.349569 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.349538 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-dshm\") pod \"premium-w6-484607-kserve-7f8d6b7d8f-fl54q\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:10.376039 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.375981 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4hlt\" (UniqueName: \"kubernetes.io/projected/6962e138-faa9-438c-9138-1133bdce1011-kube-api-access-x4hlt\") pod \"premium-w6-484607-kserve-7f8d6b7d8f-fl54q\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:10.417384 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.417358 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:10.745516 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:10.745432 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q"] Oct 30 13:04:10.749389 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:04:10.749362 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6962e138_faa9_438c_9138_1133bdce1011.slice/crio-32b492900f25285d660f326e2444ed5caabf024a506f89bfb2dd84f9414ae69c WatchSource:0}: Error finding container 32b492900f25285d660f326e2444ed5caabf024a506f89bfb2dd84f9414ae69c: Status 404 returned error can't find the container with id 32b492900f25285d660f326e2444ed5caabf024a506f89bfb2dd84f9414ae69c Oct 30 13:04:11.477030 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:11.476983 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" event={"ID":"6962e138-faa9-438c-9138-1133bdce1011","Type":"ContainerStarted","Data":"42967a7990a5cd4d4d12e4ce63cf7e8cb0076366d33efef8944759e0e8463c9b"} Oct 30 13:04:11.477482 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:11.477053 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" event={"ID":"6962e138-faa9-438c-9138-1133bdce1011","Type":"ContainerStarted","Data":"32b492900f25285d660f326e2444ed5caabf024a506f89bfb2dd84f9414ae69c"} Oct 30 13:04:11.527016 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:11.526967 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" podStartSLOduration=1.5269548130000001 podStartE2EDuration="1.526954813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:04:11.52524919 +0000 UTC m=+750.550890464" watchObservedRunningTime="2026-10-30 13:04:11.526954813 +0000 UTC m=+750.552596060" Oct 30 13:04:12.417594 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:12.417558 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:14.440872 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.440840 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg"] Oct 30 13:04:14.443817 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.443800 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:14.474735 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.474706 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-175831\"/\"premium-w5-175831-kserve-self-signed-certs\"" Oct 30 13:04:14.476888 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.476864 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg"] Oct 30 13:04:14.588731 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.588693 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-home\") pod \"premium-w5-175831-kserve-6cc7d45586-fwrbg\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:14.588980 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.588788 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-dshm\") pod \"premium-w5-175831-kserve-6cc7d45586-fwrbg\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:14.589883 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.589853 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-tls-certs\") pod \"premium-w5-175831-kserve-6cc7d45586-fwrbg\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:14.590051 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.589892 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-model-cache\") pod \"premium-w5-175831-kserve-6cc7d45586-fwrbg\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:14.590051 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.589932 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgxtx\" (UniqueName: \"kubernetes.io/projected/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-kube-api-access-mgxtx\") pod \"premium-w5-175831-kserve-6cc7d45586-fwrbg\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:14.590200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.590102 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-tmp-dir\") pod \"premium-w5-175831-kserve-6cc7d45586-fwrbg\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:14.691300 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.691198 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-tls-certs\") pod \"premium-w5-175831-kserve-6cc7d45586-fwrbg\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:14.691300 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.691239 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-model-cache\") pod \"premium-w5-175831-kserve-6cc7d45586-fwrbg\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:14.691300 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.691261 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgxtx\" (UniqueName: \"kubernetes.io/projected/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-kube-api-access-mgxtx\") pod \"premium-w5-175831-kserve-6cc7d45586-fwrbg\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:14.691588 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.691388 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-tmp-dir\") pod \"premium-w5-175831-kserve-6cc7d45586-fwrbg\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:14.691588 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.691453 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-home\") pod \"premium-w5-175831-kserve-6cc7d45586-fwrbg\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:14.691588 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.691532 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-dshm\") pod \"premium-w5-175831-kserve-6cc7d45586-fwrbg\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:14.691747 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.691718 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-model-cache\") pod \"premium-w5-175831-kserve-6cc7d45586-fwrbg\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:14.691747 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.691741 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-tmp-dir\") pod \"premium-w5-175831-kserve-6cc7d45586-fwrbg\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:14.691851 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.691806 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-home\") pod \"premium-w5-175831-kserve-6cc7d45586-fwrbg\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:14.693629 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.693605 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-dshm\") pod \"premium-w5-175831-kserve-6cc7d45586-fwrbg\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:14.693798 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.693781 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-tls-certs\") pod \"premium-w5-175831-kserve-6cc7d45586-fwrbg\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:14.716224 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.716194 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgxtx\" (UniqueName: \"kubernetes.io/projected/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-kube-api-access-mgxtx\") pod \"premium-w5-175831-kserve-6cc7d45586-fwrbg\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:14.755006 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:14.754975 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:14.880387 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:04:14.880323 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:04:15.057028 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:15.056943 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg"] Oct 30 13:04:15.060653 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:04:15.060625 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3f1a5b4_af0c_4ae3_b386_688d0cf924a5.slice/crio-db1a649312d9563adbe1ce05e06c2e13c116c70dcef6ec81384b2cd4966894a0 WatchSource:0}: Error finding container db1a649312d9563adbe1ce05e06c2e13c116c70dcef6ec81384b2cd4966894a0: Status 404 returned error can't find the container with id db1a649312d9563adbe1ce05e06c2e13c116c70dcef6ec81384b2cd4966894a0 Oct 30 13:04:15.501368 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:15.501328 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" event={"ID":"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5","Type":"ContainerStarted","Data":"e257543bc51f2701f7c36eb75e5a1c2e53cd0dfeaa769fabed96ade1638368bf"} Oct 30 13:04:15.501368 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:15.501387 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" event={"ID":"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5","Type":"ContainerStarted","Data":"db1a649312d9563adbe1ce05e06c2e13c116c70dcef6ec81384b2cd4966894a0"} Oct 30 13:04:15.545518 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:15.545455 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" podStartSLOduration=1.5454376440000002 podStartE2EDuration="1.545437644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:04:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:04:15.544586604 +0000 UTC m=+754.570227875" watchObservedRunningTime="2026-10-30 13:04:15.545437644 +0000 UTC m=+754.571078894" Oct 30 13:04:15.755950 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:15.755873 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:20.417593 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:20.417555 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:20.430590 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:20.430566 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:20.538580 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:20.538554 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:04:24.755554 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:24.755469 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:24.768044 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:24.768020 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:25.563929 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:25.563895 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:04:31.859238 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:31.859203 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld"] Oct 30 13:04:31.862035 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:31.862020 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:31.890214 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:31.890181 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld"] Oct 30 13:04:31.899224 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:31.899183 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-ac0c58\"/\"premium-w1-ac0c58-kserve-self-signed-certs\"" Oct 30 13:04:31.958784 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:31.958746 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-dshm\") pod \"premium-w1-ac0c58-kserve-557f9bcc9c-vqzld\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:31.958935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:31.958880 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmpj9\" (UniqueName: \"kubernetes.io/projected/2b878b27-465c-433d-aba4-e31849cfc14c-kube-api-access-lmpj9\") pod \"premium-w1-ac0c58-kserve-557f9bcc9c-vqzld\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:31.958935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:31.958917 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-home\") pod \"premium-w1-ac0c58-kserve-557f9bcc9c-vqzld\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:31.959010 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:31.958939 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b878b27-465c-433d-aba4-e31849cfc14c-tls-certs\") pod \"premium-w1-ac0c58-kserve-557f9bcc9c-vqzld\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:31.959114 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:31.959096 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-tmp-dir\") pod \"premium-w1-ac0c58-kserve-557f9bcc9c-vqzld\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:31.959162 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:31.959120 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-model-cache\") pod \"premium-w1-ac0c58-kserve-557f9bcc9c-vqzld\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:32.060747 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:32.060697 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-dshm\") pod \"premium-w1-ac0c58-kserve-557f9bcc9c-vqzld\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:32.060953 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:32.060808 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmpj9\" (UniqueName: \"kubernetes.io/projected/2b878b27-465c-433d-aba4-e31849cfc14c-kube-api-access-lmpj9\") pod \"premium-w1-ac0c58-kserve-557f9bcc9c-vqzld\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:32.060953 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:32.060861 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-home\") pod \"premium-w1-ac0c58-kserve-557f9bcc9c-vqzld\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:32.060953 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:32.060897 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b878b27-465c-433d-aba4-e31849cfc14c-tls-certs\") pod \"premium-w1-ac0c58-kserve-557f9bcc9c-vqzld\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:32.061171 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:32.060969 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-tmp-dir\") pod \"premium-w1-ac0c58-kserve-557f9bcc9c-vqzld\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:32.061171 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:32.060998 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-model-cache\") pod \"premium-w1-ac0c58-kserve-557f9bcc9c-vqzld\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:32.063014 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:32.061502 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-model-cache\") pod \"premium-w1-ac0c58-kserve-557f9bcc9c-vqzld\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:32.063014 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:32.062126 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-home\") pod \"premium-w1-ac0c58-kserve-557f9bcc9c-vqzld\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:32.063558 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:32.063529 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-tmp-dir\") pod \"premium-w1-ac0c58-kserve-557f9bcc9c-vqzld\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:32.065946 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:32.065902 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-dshm\") pod \"premium-w1-ac0c58-kserve-557f9bcc9c-vqzld\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:32.066265 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:32.066242 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b878b27-465c-433d-aba4-e31849cfc14c-tls-certs\") pod \"premium-w1-ac0c58-kserve-557f9bcc9c-vqzld\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:32.091853 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:32.091818 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmpj9\" (UniqueName: \"kubernetes.io/projected/2b878b27-465c-433d-aba4-e31849cfc14c-kube-api-access-lmpj9\") pod \"premium-w1-ac0c58-kserve-557f9bcc9c-vqzld\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:32.176698 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:32.176624 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:32.477478 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:32.477396 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld"] Oct 30 13:04:32.481438 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:04:32.481410 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b878b27_465c_433d_aba4_e31849cfc14c.slice/crio-cf4bdb1974d25104f6f24eff9333caab9758e77ef376b6df2a5b8b037f1fafba WatchSource:0}: Error finding container cf4bdb1974d25104f6f24eff9333caab9758e77ef376b6df2a5b8b037f1fafba: Status 404 returned error can't find the container with id cf4bdb1974d25104f6f24eff9333caab9758e77ef376b6df2a5b8b037f1fafba Oct 30 13:04:32.590661 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:32.590625 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" event={"ID":"2b878b27-465c-433d-aba4-e31849cfc14c","Type":"ContainerStarted","Data":"2078ff625415e5439f8e1f3518b20846154f80f5d309076e148861d88a25c4d8"} Oct 30 13:04:32.590661 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:32.590666 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" event={"ID":"2b878b27-465c-433d-aba4-e31849cfc14c","Type":"ContainerStarted","Data":"cf4bdb1974d25104f6f24eff9333caab9758e77ef376b6df2a5b8b037f1fafba"} Oct 30 13:04:32.634431 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:32.634366 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" podStartSLOduration=1.634347792 podStartE2EDuration="1.634347792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:04:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:04:32.63177384 +0000 UTC m=+771.657415100" watchObservedRunningTime="2026-10-30 13:04:32.634347792 +0000 UTC m=+771.659989041" Oct 30 13:04:33.177197 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:33.177126 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:42.177823 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:42.177790 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:42.190547 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:42.190523 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:42.653251 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:42.653218 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:04:44.879444 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:04:44.879410 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:04:49.975209 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:49.975174 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q"] Oct 30 13:04:49.979482 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:49.979453 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.005343 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.005309 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q"] Oct 30 13:04:50.036775 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.036740 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-workload-socket\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.036942 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.036784 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-token\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.036942 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.036804 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istiod-ca-cert\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.036942 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.036925 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-data\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.037050 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.036974 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-podinfo\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.037145 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.037132 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-credential-socket\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.037181 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.037153 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-envoy\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.037181 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.037168 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78gsn\" (UniqueName: \"kubernetes.io/projected/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-kube-api-access-78gsn\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.037257 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.037196 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-workload-certs\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.080113 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.080080 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-54994c-openshift-default-dockercfg-2qhnp\"" Oct 30 13:04:50.137984 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.137952 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-podinfo\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.138287 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.138262 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-credential-socket\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.138377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.138300 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-envoy\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.138377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.138326 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78gsn\" (UniqueName: \"kubernetes.io/projected/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-kube-api-access-78gsn\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.138486 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.138389 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-workload-certs\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.138486 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.138423 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-workload-socket\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.138594 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.138476 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-token\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.138594 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.138517 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istiod-ca-cert\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.138594 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.138551 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-data\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.138743 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.138634 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-credential-socket\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.138913 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.138889 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-workload-socket\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.139114 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.139094 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-data\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.139114 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.139100 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-workload-certs\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.139254 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.139208 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istiod-ca-cert\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.140507 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.140486 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-podinfo\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.140592 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.140490 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-envoy\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.165791 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.165757 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-token\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.166035 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.166016 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78gsn\" (UniqueName: \"kubernetes.io/projected/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-kube-api-access-78gsn\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.296278 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.296191 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:50.622012 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.621845 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q"] Oct 30 13:04:50.624873 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:04:50.624849 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48dfe4a1_9d55_4f04_812a_1d0084f01c7e.slice/crio-cd867d6c68659bac758b53b05172a7a1b63de19251c4c628866946f133735c89 WatchSource:0}: Error finding container cd867d6c68659bac758b53b05172a7a1b63de19251c4c628866946f133735c89: Status 404 returned error can't find the container with id cd867d6c68659bac758b53b05172a7a1b63de19251c4c628866946f133735c89 Oct 30 13:04:50.627303 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.627270 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:04:50.627414 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.627336 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:04:50.627414 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.627366 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:04:50.685390 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:50.685360 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" event={"ID":"48dfe4a1-9d55-4f04-812a-1d0084f01c7e","Type":"ContainerStarted","Data":"cd867d6c68659bac758b53b05172a7a1b63de19251c4c628866946f133735c89"} Oct 30 13:04:51.701226 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:51.701187 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" event={"ID":"48dfe4a1-9d55-4f04-812a-1d0084f01c7e","Type":"ContainerStarted","Data":"1d7914ec89b193b3f4151530bab81d31d1928575b9b3d5e0e6dcefed7991df7b"} Oct 30 13:04:51.771108 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:51.770834 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" podStartSLOduration=2.770817398 podStartE2EDuration="2.770817398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:04:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:04:51.757803561 +0000 UTC m=+790.783444831" watchObservedRunningTime="2026-10-30 13:04:51.770817398 +0000 UTC m=+790.796458647" Oct 30 13:04:52.296344 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:52.296305 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:52.297694 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:52.297662 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Oct 30 13:04:52.297825 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:52.297709 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" podUID="48dfe4a1-9d55-4f04-812a-1d0084f01c7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Oct 30 13:04:53.302965 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:53.302460 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Oct 30 13:04:53.302965 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:53.302556 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" podUID="48dfe4a1-9d55-4f04-812a-1d0084f01c7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Oct 30 13:04:54.297613 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:54.297570 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Oct 30 13:04:54.297855 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:54.297655 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" podUID="48dfe4a1-9d55-4f04-812a-1d0084f01c7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Oct 30 13:04:55.297364 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:55.297327 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Oct 30 13:04:55.297864 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:55.297398 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" podUID="48dfe4a1-9d55-4f04-812a-1d0084f01c7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Oct 30 13:04:56.297276 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:56.297243 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Oct 30 13:04:56.297481 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:56.297307 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" podUID="48dfe4a1-9d55-4f04-812a-1d0084f01c7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Oct 30 13:04:57.296939 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:57.296899 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Oct 30 13:04:57.297138 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:57.296967 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" podUID="48dfe4a1-9d55-4f04-812a-1d0084f01c7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Oct 30 13:04:58.421878 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:58.421844 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:58.422632 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:58.422195 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:04:58.423312 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:04:58.423265 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:05:14.880051 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:05:14.880013 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:05:19.838254 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:19.838210 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4"] Oct 30 13:05:19.842216 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:19.842185 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:19.865507 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:19.865478 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-3048f9-openshift-default-dockercfg-b5fdc\"" Oct 30 13:05:19.884296 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:19.884262 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4"] Oct 30 13:05:19.922977 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:19.922934 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-credential-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:19.928544 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:19.928514 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dghcb\" (UniqueName: \"kubernetes.io/projected/184f5837-5aee-46b3-856c-4c3db17a7877-kube-api-access-dghcb\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:19.928799 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:19.928776 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-workload-certs\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:19.928938 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:19.928921 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/184f5837-5aee-46b3-856c-4c3db17a7877-istio-token\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:19.929172 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:19.929133 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/184f5837-5aee-46b3-856c-4c3db17a7877-istio-podinfo\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:19.929381 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:19.929339 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-istio-envoy\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:19.929590 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:19.929571 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-workload-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:19.929758 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:19.929742 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-istio-data\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:19.929889 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:19.929874 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/184f5837-5aee-46b3-856c-4c3db17a7877-istiod-ca-cert\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:20.031488 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.031449 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-workload-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:20.031790 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.031766 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-istio-data\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:20.031954 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.031921 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/184f5837-5aee-46b3-856c-4c3db17a7877-istiod-ca-cert\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:20.032116 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.031985 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-credential-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:20.032116 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.032033 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dghcb\" (UniqueName: \"kubernetes.io/projected/184f5837-5aee-46b3-856c-4c3db17a7877-kube-api-access-dghcb\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:20.032116 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.032105 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-workload-certs\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:20.032314 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.032131 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/184f5837-5aee-46b3-856c-4c3db17a7877-istio-token\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:20.032314 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.032194 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/184f5837-5aee-46b3-856c-4c3db17a7877-istio-podinfo\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:20.032314 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.032228 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-istio-data\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:20.032314 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.032268 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-istio-envoy\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:20.032625 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.031929 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-workload-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:20.033270 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.033250 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-workload-certs\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:20.033705 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.033682 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/184f5837-5aee-46b3-856c-4c3db17a7877-istiod-ca-cert\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:20.035436 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.035145 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-istio-envoy\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:20.046209 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.035444 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-credential-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:20.046209 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.036203 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/184f5837-5aee-46b3-856c-4c3db17a7877-istio-podinfo\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:20.059612 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.059551 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dghcb\" (UniqueName: \"kubernetes.io/projected/184f5837-5aee-46b3-856c-4c3db17a7877-kube-api-access-dghcb\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:20.059738 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.059718 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/184f5837-5aee-46b3-856c-4c3db17a7877-istio-token\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:20.161158 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.160860 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:20.523318 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.521983 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4"] Oct 30 13:05:20.530435 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.530393 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:05:20.530546 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.530483 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:05:20.530546 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.530532 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:05:20.924169 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.918365 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" event={"ID":"184f5837-5aee-46b3-856c-4c3db17a7877","Type":"ContainerStarted","Data":"3e6b43ce3f5902f8195efed31f769bd2ac7cec5d28d618c2e367e91202a0ef99"} Oct 30 13:05:20.924169 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.918453 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" event={"ID":"184f5837-5aee-46b3-856c-4c3db17a7877","Type":"ContainerStarted","Data":"f025b6076a887eab0c15c8493833a884902c5f8b7c29cfcae61481ea125b591c"} Oct 30 13:05:20.980090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:20.977826 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" podStartSLOduration=1.97780816 podStartE2EDuration="1.97780816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:05:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:05:20.974598999 +0000 UTC m=+820.000240273" watchObservedRunningTime="2026-10-30 13:05:20.97780816 +0000 UTC m=+820.003449409" Oct 30 13:05:21.162089 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:21.162044 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:21.163470 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:21.163442 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Oct 30 13:05:21.163610 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:21.163488 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" podUID="184f5837-5aee-46b3-856c-4c3db17a7877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Oct 30 13:05:22.161792 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:22.161760 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Oct 30 13:05:22.162276 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:22.162245 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" podUID="184f5837-5aee-46b3-856c-4c3db17a7877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Oct 30 13:05:23.161224 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:23.161183 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Oct 30 13:05:23.161423 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:23.161244 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" podUID="184f5837-5aee-46b3-856c-4c3db17a7877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Oct 30 13:05:24.161243 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:24.161212 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Oct 30 13:05:24.161722 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:24.161266 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" podUID="184f5837-5aee-46b3-856c-4c3db17a7877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Oct 30 13:05:25.161434 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:25.161400 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Oct 30 13:05:25.161940 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:25.161464 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" podUID="184f5837-5aee-46b3-856c-4c3db17a7877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Oct 30 13:05:26.161203 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:26.161167 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Oct 30 13:05:26.161493 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:26.161235 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" podUID="184f5837-5aee-46b3-856c-4c3db17a7877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Oct 30 13:05:27.162112 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:27.162082 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Oct 30 13:05:27.162479 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:27.162133 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" podUID="184f5837-5aee-46b3-856c-4c3db17a7877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Oct 30 13:05:28.165455 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:28.165427 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:28.165947 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:28.165930 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:28.166615 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:28.166597 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:05:35.017032 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.016997 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn"] Oct 30 13:05:35.019951 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.019934 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.035920 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.035898 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-bb71e2-openshift-default-dockercfg-qmj6k\"" Oct 30 13:05:35.063807 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.063754 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn"] Oct 30 13:05:35.127840 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.127805 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-data\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.128023 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.127854 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzntb\" (UniqueName: \"kubernetes.io/projected/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-kube-api-access-rzntb\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.128023 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.127912 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-workload-socket\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.128023 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.127954 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-workload-certs\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.128023 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.128005 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-credential-socket\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.128269 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.128051 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istiod-ca-cert\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.128269 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.128107 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-envoy\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.128269 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.128167 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-podinfo\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.128417 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.128307 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-token\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.228863 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.228832 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzntb\" (UniqueName: \"kubernetes.io/projected/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-kube-api-access-rzntb\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.229046 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.228881 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-workload-socket\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.229046 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.228904 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-workload-certs\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.229046 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.228930 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-credential-socket\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.229046 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.228953 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istiod-ca-cert\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.229295 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.229060 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-envoy\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.229295 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.229145 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-podinfo\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.229295 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.229227 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-token\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.229455 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.229309 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-data\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.229455 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.229435 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-credential-socket\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.229558 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.229505 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-workload-certs\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.229663 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.229643 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-workload-socket\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.229723 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.229658 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-data\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.229826 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.229805 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istiod-ca-cert\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.231479 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.231457 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-envoy\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.231719 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.231697 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-podinfo\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.259119 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.259093 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-token\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.259389 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.259371 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzntb\" (UniqueName: \"kubernetes.io/projected/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-kube-api-access-rzntb\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.297220 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.297148 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8"] Oct 30 13:05:35.301035 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.301015 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.335660 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.334533 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8"] Oct 30 13:05:35.335660 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.335264 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:35.335660 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.335518 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-envoy\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.335660 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.335547 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-podinfo\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.335660 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.335582 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-workload-socket\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.336000 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.335693 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4kvp\" (UniqueName: \"kubernetes.io/projected/a2dc4823-8550-4c35-a226-6440fbbffbc5-kube-api-access-h4kvp\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.336000 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.335735 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2dc4823-8550-4c35-a226-6440fbbffbc5-istiod-ca-cert\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.336000 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.335832 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-token\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.336000 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.335886 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-workload-certs\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.336000 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.335950 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-credential-socket\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.336000 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.335972 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-data\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.437270 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.437234 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-envoy\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.437427 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.437282 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-podinfo\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.438804 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.437519 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-workload-socket\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.438804 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.437556 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4kvp\" (UniqueName: \"kubernetes.io/projected/a2dc4823-8550-4c35-a226-6440fbbffbc5-kube-api-access-h4kvp\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.438804 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.437581 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2dc4823-8550-4c35-a226-6440fbbffbc5-istiod-ca-cert\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.438804 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.437625 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-token\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.438804 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.437651 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-workload-certs\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.438804 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.437721 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-credential-socket\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.438804 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.437770 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-data\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.438804 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.438240 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-data\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.438804 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.438360 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-workload-socket\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.438804 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.438652 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2dc4823-8550-4c35-a226-6440fbbffbc5-istiod-ca-cert\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.438804 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.438762 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-workload-certs\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.439240 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.438829 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-credential-socket\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.440293 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.440247 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-podinfo\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.440970 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.440947 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-envoy\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.462298 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.462265 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-token\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.462407 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.462325 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4kvp\" (UniqueName: \"kubernetes.io/projected/a2dc4823-8550-4c35-a226-6440fbbffbc5-kube-api-access-h4kvp\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.616769 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.616735 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:35.668345 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.668313 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn"] Oct 30 13:05:35.672517 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:05:35.672479 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17f3c759_be0a_4f0d_b78a_0e4cbe8adfa2.slice/crio-713c6258000cdae307c1eaccbfc33a8fa7bc604fcf84de0c5a17674fcff4be84 WatchSource:0}: Error finding container 713c6258000cdae307c1eaccbfc33a8fa7bc604fcf84de0c5a17674fcff4be84: Status 404 returned error can't find the container with id 713c6258000cdae307c1eaccbfc33a8fa7bc604fcf84de0c5a17674fcff4be84 Oct 30 13:05:35.684163 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.676329 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:05:35.684163 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.676393 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:05:35.684163 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.676420 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:05:35.924306 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.924221 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8"] Oct 30 13:05:35.927979 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:05:35.927951 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2dc4823_8550_4c35_a226_6440fbbffbc5.slice/crio-ce073ed4d6089ea28daef2fda4dbfbb3476f63c88dea7df5cfbc57715ca4dca1 WatchSource:0}: Error finding container ce073ed4d6089ea28daef2fda4dbfbb3476f63c88dea7df5cfbc57715ca4dca1: Status 404 returned error can't find the container with id ce073ed4d6089ea28daef2fda4dbfbb3476f63c88dea7df5cfbc57715ca4dca1 Oct 30 13:05:35.930627 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.930596 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:05:35.930696 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.930661 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:05:35.930696 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:35.930692 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:05:36.021461 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:36.021416 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" event={"ID":"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2","Type":"ContainerStarted","Data":"353b8134f8946b557d65acf9cec18d086032d1424a8fdbd1a8a0c8c7ce2a3cf3"} Oct 30 13:05:36.022685 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:36.021483 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" event={"ID":"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2","Type":"ContainerStarted","Data":"713c6258000cdae307c1eaccbfc33a8fa7bc604fcf84de0c5a17674fcff4be84"} Oct 30 13:05:36.024014 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:36.023862 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" event={"ID":"a2dc4823-8550-4c35-a226-6440fbbffbc5","Type":"ContainerStarted","Data":"9dd56de4da3541e23ca5549bbf7d8bfb78f7e52af03417554c1886d0a3a9b904"} Oct 30 13:05:36.024014 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:36.023921 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" event={"ID":"a2dc4823-8550-4c35-a226-6440fbbffbc5","Type":"ContainerStarted","Data":"ce073ed4d6089ea28daef2fda4dbfbb3476f63c88dea7df5cfbc57715ca4dca1"} Oct 30 13:05:36.072992 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:36.072934 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" podStartSLOduration=2.072917058 podStartE2EDuration="2.072917058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:05:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:05:36.070081908 +0000 UTC m=+835.095723185" watchObservedRunningTime="2026-10-30 13:05:36.072917058 +0000 UTC m=+835.098558308" Oct 30 13:05:36.130155 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:36.130062 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" podStartSLOduration=1.130043442 podStartE2EDuration="1.130043442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:05:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:05:36.127923418 +0000 UTC m=+835.153564683" watchObservedRunningTime="2026-10-30 13:05:36.130043442 +0000 UTC m=+835.155684692" Oct 30 13:05:36.336318 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:36.336280 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:36.337687 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:36.337665 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Oct 30 13:05:36.337784 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:36.337709 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" podUID="17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Oct 30 13:05:36.620148 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:36.617431 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:36.624253 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:36.622192 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Oct 30 13:05:36.624253 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:36.622258 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" podUID="a2dc4823-8550-4c35-a226-6440fbbffbc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Oct 30 13:05:37.335939 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:37.335907 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Oct 30 13:05:37.336321 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:37.335953 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" podUID="17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Oct 30 13:05:37.619149 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:37.618436 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Oct 30 13:05:37.619149 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:37.618512 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" podUID="a2dc4823-8550-4c35-a226-6440fbbffbc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Oct 30 13:05:38.336187 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:38.336153 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Oct 30 13:05:38.336600 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:38.336203 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" podUID="17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Oct 30 13:05:38.620143 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:38.619436 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Oct 30 13:05:38.620143 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:38.619513 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" podUID="a2dc4823-8550-4c35-a226-6440fbbffbc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Oct 30 13:05:39.337092 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:39.336345 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Oct 30 13:05:39.337092 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:39.336406 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" podUID="17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Oct 30 13:05:39.627147 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:39.626540 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Oct 30 13:05:39.627147 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:39.626619 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" podUID="a2dc4823-8550-4c35-a226-6440fbbffbc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Oct 30 13:05:40.336745 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:40.336689 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Oct 30 13:05:40.336957 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:40.336764 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" podUID="17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Oct 30 13:05:40.621148 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:40.617482 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Oct 30 13:05:40.621148 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:40.617550 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" podUID="a2dc4823-8550-4c35-a226-6440fbbffbc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Oct 30 13:05:41.336516 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:41.336480 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Oct 30 13:05:41.336700 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:41.336540 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" podUID="17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Oct 30 13:05:41.620571 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:41.620475 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Oct 30 13:05:41.620830 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:41.620800 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" podUID="a2dc4823-8550-4c35-a226-6440fbbffbc5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Oct 30 13:05:42.511677 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:42.511646 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:42.512924 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:42.512897 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:42.513756 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:42.513736 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:05:42.905264 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:42.905226 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:42.906093 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:42.906038 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:42.906833 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:42.906812 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:05:44.323784 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.323742 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6"] Oct 30 13:05:44.326772 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.326756 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:44.358179 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.358141 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-484607\"/\"distinct-w6-484607-kserve-self-signed-certs\"" Oct 30 13:05:44.361887 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.361857 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6"] Oct 30 13:05:44.409006 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.408975 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-tls-certs\") pod \"distinct-w6-484607-kserve-7585c5f98d-8zzv6\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:44.409168 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.409056 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjmhq\" (UniqueName: \"kubernetes.io/projected/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-kube-api-access-vjmhq\") pod \"distinct-w6-484607-kserve-7585c5f98d-8zzv6\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:44.409220 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.409205 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-tmp-dir\") pod \"distinct-w6-484607-kserve-7585c5f98d-8zzv6\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:44.409322 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.409232 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-dshm\") pod \"distinct-w6-484607-kserve-7585c5f98d-8zzv6\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:44.409322 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.409253 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-model-cache\") pod \"distinct-w6-484607-kserve-7585c5f98d-8zzv6\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:44.409481 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.409364 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-home\") pod \"distinct-w6-484607-kserve-7585c5f98d-8zzv6\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:44.514660 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.514177 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjmhq\" (UniqueName: \"kubernetes.io/projected/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-kube-api-access-vjmhq\") pod \"distinct-w6-484607-kserve-7585c5f98d-8zzv6\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:44.514660 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.514287 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-tmp-dir\") pod \"distinct-w6-484607-kserve-7585c5f98d-8zzv6\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:44.514660 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.514320 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-dshm\") pod \"distinct-w6-484607-kserve-7585c5f98d-8zzv6\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:44.514660 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.514349 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-model-cache\") pod \"distinct-w6-484607-kserve-7585c5f98d-8zzv6\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:44.514660 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.514430 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-home\") pod \"distinct-w6-484607-kserve-7585c5f98d-8zzv6\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:44.514660 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.514469 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-tls-certs\") pod \"distinct-w6-484607-kserve-7585c5f98d-8zzv6\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:44.515544 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.515521 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-model-cache\") pod \"distinct-w6-484607-kserve-7585c5f98d-8zzv6\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:44.515988 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.515962 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-home\") pod \"distinct-w6-484607-kserve-7585c5f98d-8zzv6\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:44.522602 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.516643 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-tmp-dir\") pod \"distinct-w6-484607-kserve-7585c5f98d-8zzv6\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:44.534551 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.534521 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-dshm\") pod \"distinct-w6-484607-kserve-7585c5f98d-8zzv6\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:44.542263 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.542235 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjmhq\" (UniqueName: \"kubernetes.io/projected/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-kube-api-access-vjmhq\") pod \"distinct-w6-484607-kserve-7585c5f98d-8zzv6\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:44.546089 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.544339 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-tls-certs\") pod \"distinct-w6-484607-kserve-7585c5f98d-8zzv6\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:44.638992 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.638883 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:44.880197 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:05:44.880156 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:05:44.946639 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:44.946557 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6"] Oct 30 13:05:44.957355 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:05:44.957327 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6ce39f3_c5ad_4c48_9a01_84123c6c7ab2.slice/crio-fdd32517a555bf8827549b4de8cf553a4a29bbf1a50b117fd907df1d00a55c89 WatchSource:0}: Error finding container fdd32517a555bf8827549b4de8cf553a4a29bbf1a50b117fd907df1d00a55c89: Status 404 returned error can't find the container with id fdd32517a555bf8827549b4de8cf553a4a29bbf1a50b117fd907df1d00a55c89 Oct 30 13:05:45.077516 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:45.077479 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" event={"ID":"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2","Type":"ContainerStarted","Data":"b8c9908df3066256bfac36e26023aab46d96c34937e80a47b502696eb3805322"} Oct 30 13:05:45.077698 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:45.077526 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" event={"ID":"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2","Type":"ContainerStarted","Data":"fdd32517a555bf8827549b4de8cf553a4a29bbf1a50b117fd907df1d00a55c89"} Oct 30 13:05:45.121109 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:45.121040 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" podStartSLOduration=1.121022701 podStartE2EDuration="1.121022701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:05:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:05:45.118533237 +0000 UTC m=+844.144174512" watchObservedRunningTime="2026-10-30 13:05:45.121022701 +0000 UTC m=+844.146663929" Oct 30 13:05:45.639828 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:45.639794 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:49.167188 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.167152 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx"] Oct 30 13:05:49.172586 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.170312 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.189174 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.189140 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-484607-openshift-default-dockercfg-ggcps\"" Oct 30 13:05:49.204426 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.204030 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx"] Oct 30 13:05:49.270819 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.270781 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-istio-data\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.270982 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.270824 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-credential-socket\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.270982 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.270874 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-workload-certs\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.270982 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.270899 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q22wf\" (UniqueName: \"kubernetes.io/projected/88f14514-918e-43a1-a097-c236701d22f3-kube-api-access-q22wf\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.270982 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.270944 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-istio-envoy\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.271246 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.270991 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88f14514-918e-43a1-a097-c236701d22f3-istio-token\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.271246 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.271050 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88f14514-918e-43a1-a097-c236701d22f3-istiod-ca-cert\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.271246 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.271119 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88f14514-918e-43a1-a097-c236701d22f3-istio-podinfo\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.271246 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.271223 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-workload-socket\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.372276 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.372232 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-workload-certs\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.372276 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.372276 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q22wf\" (UniqueName: \"kubernetes.io/projected/88f14514-918e-43a1-a097-c236701d22f3-kube-api-access-q22wf\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.372532 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.372323 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-istio-envoy\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.372532 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.372369 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88f14514-918e-43a1-a097-c236701d22f3-istio-token\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.372532 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.372425 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88f14514-918e-43a1-a097-c236701d22f3-istiod-ca-cert\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.372532 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.372474 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88f14514-918e-43a1-a097-c236701d22f3-istio-podinfo\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.372731 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.372574 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-workload-socket\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.372731 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.372642 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-istio-data\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.372731 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.372672 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-credential-socket\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.374180 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.373127 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-credential-socket\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.374180 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.373558 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-workload-certs\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.374180 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.373776 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88f14514-918e-43a1-a097-c236701d22f3-istiod-ca-cert\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.374733 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.374710 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-workload-socket\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.375136 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.375114 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-istio-data\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.378218 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.378177 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-istio-envoy\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.378397 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.378372 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88f14514-918e-43a1-a097-c236701d22f3-istio-podinfo\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.399593 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.397818 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88f14514-918e-43a1-a097-c236701d22f3-istio-token\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.399593 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.398957 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q22wf\" (UniqueName: \"kubernetes.io/projected/88f14514-918e-43a1-a097-c236701d22f3-kube-api-access-q22wf\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.531810 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.531722 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:49.554523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.554493 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq"] Oct 30 13:05:49.557772 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.557754 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.574442 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.574406 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-175831-openshift-default-dockercfg-whpw8\"" Oct 30 13:05:49.589303 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.587190 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq"] Oct 30 13:05:49.676130 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.676094 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/819605a7-0156-4e46-922c-26d314885598-istiod-ca-cert\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.676321 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.676159 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-istio-envoy\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.676392 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.676307 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/819605a7-0156-4e46-922c-26d314885598-istio-token\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.676392 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.676379 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-workload-socket\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.676529 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.676507 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-credential-socket\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.676586 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.676563 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glrn4\" (UniqueName: \"kubernetes.io/projected/819605a7-0156-4e46-922c-26d314885598-kube-api-access-glrn4\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.676664 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.676649 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-workload-certs\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.676718 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.676679 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-istio-data\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.676784 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.676767 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/819605a7-0156-4e46-922c-26d314885598-istio-podinfo\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.762026 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.761994 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp"] Oct 30 13:05:49.764706 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.764688 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:49.777739 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.777712 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/819605a7-0156-4e46-922c-26d314885598-istio-podinfo\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.777845 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.777774 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/819605a7-0156-4e46-922c-26d314885598-istiod-ca-cert\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.777845 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.777819 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-istio-envoy\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.777917 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.777860 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/819605a7-0156-4e46-922c-26d314885598-istio-token\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.777974 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.777926 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-workload-socket\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.778643 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.778037 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-credential-socket\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.778643 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.778108 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glrn4\" (UniqueName: \"kubernetes.io/projected/819605a7-0156-4e46-922c-26d314885598-kube-api-access-glrn4\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.778643 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.778166 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-workload-certs\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.778643 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.778195 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-istio-data\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.778643 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.778409 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-workload-socket\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.778643 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.778582 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-istio-data\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.778929 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.778650 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-workload-certs\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.778929 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.778674 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-credential-socket\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.779002 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.778986 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/819605a7-0156-4e46-922c-26d314885598-istiod-ca-cert\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.780212 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.780192 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-istio-envoy\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.780534 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.780517 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/819605a7-0156-4e46-922c-26d314885598-istio-podinfo\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.782298 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.782245 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-42ee9d-openshift-default-dockercfg-sb4pn\"" Oct 30 13:05:49.790103 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.789230 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp"] Oct 30 13:05:49.802786 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.802761 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glrn4\" (UniqueName: \"kubernetes.io/projected/819605a7-0156-4e46-922c-26d314885598-kube-api-access-glrn4\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.803256 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.803236 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/819605a7-0156-4e46-922c-26d314885598-istio-token\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.864916 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.864833 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx"] Oct 30 13:05:49.878139 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:05:49.872320 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88f14514_918e_43a1_a097_c236701d22f3.slice/crio-d873aed3ca6d0ac5657846de971cc2abafd18dfbe2de88001fcab5399ad8edb3 WatchSource:0}: Error finding container d873aed3ca6d0ac5657846de971cc2abafd18dfbe2de88001fcab5399ad8edb3: Status 404 returned error can't find the container with id d873aed3ca6d0ac5657846de971cc2abafd18dfbe2de88001fcab5399ad8edb3 Oct 30 13:05:49.878139 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.872527 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:49.879004 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.878970 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-workload-certs\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:49.879247 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.879229 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-token\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:49.879431 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.879416 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-envoy\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:49.879583 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.879562 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-data\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:49.895081 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.879675 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm7w4\" (UniqueName: \"kubernetes.io/projected/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-kube-api-access-sm7w4\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:49.895495 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.886008 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:05:49.895595 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.895565 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:05:49.895672 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.895625 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:05:49.896120 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.896082 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-podinfo\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:49.896247 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.896143 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istiod-ca-cert\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:49.896247 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.896194 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-workload-socket\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:49.896362 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.896247 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-credential-socket\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:49.999442 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.997800 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-podinfo\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:49.999442 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.997835 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istiod-ca-cert\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:49.999442 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.997871 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-workload-socket\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:49.999442 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.997900 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-credential-socket\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:49.999442 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.997965 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-workload-certs\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:49.999442 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.997986 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-token\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:49.999442 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.998045 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-envoy\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:49.999442 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.998133 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-data\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:49.999442 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.998181 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm7w4\" (UniqueName: \"kubernetes.io/projected/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-kube-api-access-sm7w4\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:49.999442 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.998456 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-workload-socket\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:49.999442 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.999008 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-workload-certs\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:49.999442 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.999238 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-credential-socket\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:50.000109 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.999700 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istiod-ca-cert\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:50.000109 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:49.999940 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-data\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:50.002223 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:50.002191 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-envoy\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:50.002734 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:50.002700 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-podinfo\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:50.023012 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:50.022743 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-token\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:50.023012 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:50.022960 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm7w4\" (UniqueName: \"kubernetes.io/projected/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-kube-api-access-sm7w4\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:50.077948 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:50.077917 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:50.111714 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:50.111678 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" event={"ID":"88f14514-918e-43a1-a097-c236701d22f3","Type":"ContainerStarted","Data":"138ff646b89ea08b16c04bc1f6eb11a2f8063004998d271c0bb497c6bbcf8beb"} Oct 30 13:05:50.111881 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:50.111732 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" event={"ID":"88f14514-918e-43a1-a097-c236701d22f3","Type":"ContainerStarted","Data":"d873aed3ca6d0ac5657846de971cc2abafd18dfbe2de88001fcab5399ad8edb3"} Oct 30 13:05:50.162568 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:50.162512 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" podStartSLOduration=1.162492344 podStartE2EDuration="1.162492344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:05:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:05:50.158538805 +0000 UTC m=+849.184180072" watchObservedRunningTime="2026-10-30 13:05:50.162492344 +0000 UTC m=+849.188133595" Oct 30 13:05:50.219983 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:50.219948 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq"] Oct 30 13:05:50.225726 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:05:50.225698 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod819605a7_0156_4e46_922c_26d314885598.slice/crio-c9107d3c665455b2619fa74feb0ac60ca17c775810c5aaec73c2473d98be6440 WatchSource:0}: Error finding container c9107d3c665455b2619fa74feb0ac60ca17c775810c5aaec73c2473d98be6440: Status 404 returned error can't find the container with id c9107d3c665455b2619fa74feb0ac60ca17c775810c5aaec73c2473d98be6440 Oct 30 13:05:50.228712 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:50.228676 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:05:50.228825 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:50.228753 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:05:50.228825 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:50.228800 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:05:50.414024 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:50.413902 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp"] Oct 30 13:05:50.415233 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:50.415195 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:05:50.415352 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:50.415270 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:05:50.415352 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:50.415313 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:05:50.532591 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:50.532533 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:50.533627 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:50.533589 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Oct 30 13:05:50.533885 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:50.533839 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" podUID="88f14514-918e-43a1-a097-c236701d22f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Oct 30 13:05:51.128226 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:51.128186 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" event={"ID":"0e89ac39-4946-4c8a-bbc0-43c2f8487aec","Type":"ContainerStarted","Data":"f98fa61e5489a42ad058e2b36ad9cb9383b30e32dc86bb6cd84155ca5ef05585"} Oct 30 13:05:51.128564 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:51.128538 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" event={"ID":"0e89ac39-4946-4c8a-bbc0-43c2f8487aec","Type":"ContainerStarted","Data":"ece7946a2c7bd1837acf17b01bfc565300007ae21a82d405f4a2ca401b038bcc"} Oct 30 13:05:51.133538 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:51.133507 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" event={"ID":"819605a7-0156-4e46-922c-26d314885598","Type":"ContainerStarted","Data":"8692fc5e14564eeb40853787bd8c729343ddbb4afbf0856cddad3e3a0b1b6667"} Oct 30 13:05:51.133771 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:51.133747 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" event={"ID":"819605a7-0156-4e46-922c-26d314885598","Type":"ContainerStarted","Data":"c9107d3c665455b2619fa74feb0ac60ca17c775810c5aaec73c2473d98be6440"} Oct 30 13:05:51.178942 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:51.178875 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" podStartSLOduration=2.178853289 podStartE2EDuration="2.178853289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:05:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:05:51.176572283 +0000 UTC m=+850.202213554" watchObservedRunningTime="2026-10-30 13:05:51.178853289 +0000 UTC m=+850.204494539" Oct 30 13:05:51.537155 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:51.536469 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Oct 30 13:05:51.537155 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:51.536532 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" podUID="88f14514-918e-43a1-a097-c236701d22f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Oct 30 13:05:51.876091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:51.875707 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:51.876091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:51.875846 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Oct 30 13:05:51.876091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:51.875895 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" podUID="819605a7-0156-4e46-922c-26d314885598" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Oct 30 13:05:52.088173 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:52.087445 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:52.088173 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:52.087585 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 30 13:05:52.088173 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:52.087625 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" podUID="0e89ac39-4946-4c8a-bbc0-43c2f8487aec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 30 13:05:52.533091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:52.532130 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Oct 30 13:05:52.533091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:52.532186 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" podUID="88f14514-918e-43a1-a097-c236701d22f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Oct 30 13:05:52.889099 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:52.888187 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Oct 30 13:05:52.889099 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:52.888250 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" podUID="819605a7-0156-4e46-922c-26d314885598" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Oct 30 13:05:53.079546 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:53.079509 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 30 13:05:53.079821 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:53.079789 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" podUID="0e89ac39-4946-4c8a-bbc0-43c2f8487aec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 30 13:05:53.533857 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:53.533409 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Oct 30 13:05:53.533857 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:53.533472 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" podUID="88f14514-918e-43a1-a097-c236701d22f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Oct 30 13:05:53.875098 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:53.874470 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Oct 30 13:05:53.875098 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:53.874531 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" podUID="819605a7-0156-4e46-922c-26d314885598" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Oct 30 13:05:54.094096 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:54.093208 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 30 13:05:54.094096 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:54.093293 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" podUID="0e89ac39-4946-4c8a-bbc0-43c2f8487aec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 30 13:05:54.533452 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:54.533415 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Oct 30 13:05:54.534118 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:54.533692 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" podUID="88f14514-918e-43a1-a097-c236701d22f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Oct 30 13:05:54.641100 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:54.640176 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:54.662223 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:54.662192 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:54.709933 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:54.709870 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" podStartSLOduration=5.709849698 podStartE2EDuration="5.709849698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:05:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:05:51.237123451 +0000 UTC m=+850.262764701" watchObservedRunningTime="2026-10-30 13:05:54.709849698 +0000 UTC m=+853.735490948" Oct 30 13:05:54.876094 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:54.874522 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Oct 30 13:05:54.876094 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:54.874597 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" podUID="819605a7-0156-4e46-922c-26d314885598" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Oct 30 13:05:55.084093 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:55.080560 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 30 13:05:55.084093 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:55.080619 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" podUID="0e89ac39-4946-4c8a-bbc0-43c2f8487aec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 30 13:05:55.479863 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:55.479586 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:05:55.533816 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:55.533392 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Oct 30 13:05:55.533816 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:55.533451 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" podUID="88f14514-918e-43a1-a097-c236701d22f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Oct 30 13:05:55.883087 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:55.878535 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Oct 30 13:05:55.883087 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:55.878617 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" podUID="819605a7-0156-4e46-922c-26d314885598" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Oct 30 13:05:56.092344 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:56.086465 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 30 13:05:56.092344 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:56.086524 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" podUID="0e89ac39-4946-4c8a-bbc0-43c2f8487aec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 30 13:05:56.545106 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:56.544250 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Oct 30 13:05:56.545106 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:56.544314 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" podUID="88f14514-918e-43a1-a097-c236701d22f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Oct 30 13:05:56.874638 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:56.874595 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Oct 30 13:05:56.874843 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:56.874662 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" podUID="819605a7-0156-4e46-922c-26d314885598" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Oct 30 13:05:57.078492 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:57.078442 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 30 13:05:57.078789 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:57.078525 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" podUID="0e89ac39-4946-4c8a-bbc0-43c2f8487aec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 30 13:05:57.539291 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:57.539251 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:57.540136 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:57.540114 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:57.542426 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:57.542100 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:05:58.033329 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:58.033288 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:58.034379 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:58.034350 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:58.035594 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:58.035572 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:05:58.086879 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:58.086849 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:58.087185 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:58.087161 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:05:58.088111 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:05:58.088092 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:06:14.878533 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:06:14.878492 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:06:41.172093 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:41.172039 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/2.log" Oct 30 13:06:41.174187 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:41.174159 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/2.log" Oct 30 13:06:44.878631 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:06:44.878592 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:06:49.351329 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.351286 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg"] Oct 30 13:06:49.356016 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.355982 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.381116 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.381084 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg"] Oct 30 13:06:49.385614 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.385586 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25"] Oct 30 13:06:49.389608 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.389588 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.413871 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.413845 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25"] Oct 30 13:06:49.481012 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.480979 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf2kv\" (UniqueName: \"kubernetes.io/projected/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-kube-api-access-tf2kv\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.481173 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.481046 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-podinfo\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.481252 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.481169 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-data\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.481252 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.481196 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-credential-socket\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.481252 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.481227 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-envoy\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.481371 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.481321 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-workload-socket\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.481371 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.481347 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istiod-ca-cert\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.481458 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.481377 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-workload-certs\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.481519 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.481486 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-token\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.582401 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.582363 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istiod-ca-cert\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.582401 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.582405 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-envoy\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.582624 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.582426 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-data\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.582624 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.582529 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-data\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.582624 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.582556 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-credential-socket\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.582624 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.582581 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-workload-socket\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.582624 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.582605 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-token\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.582866 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.582737 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-envoy\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.582866 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.582767 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-workload-certs\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.582866 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.582800 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-workload-socket\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.582866 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.582819 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istiod-ca-cert\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.583096 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.582933 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-data\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.583096 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.582968 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-workload-certs\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.583096 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.583019 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-token\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.583096 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.583023 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-credential-socket\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.583096 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.583049 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-podinfo\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.583363 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.583145 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf2kv\" (UniqueName: \"kubernetes.io/projected/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-kube-api-access-tf2kv\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.583363 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.583215 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-workload-socket\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.583363 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.583264 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-workload-certs\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.583363 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.583270 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-credential-socket\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.583363 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.583349 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-podinfo\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.583566 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.583387 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcv92\" (UniqueName: \"kubernetes.io/projected/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-kube-api-access-dcv92\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.583566 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.583533 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istiod-ca-cert\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.585031 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.585007 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-envoy\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.585517 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.585499 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-podinfo\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.627938 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.627884 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-token\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.628140 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.628115 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf2kv\" (UniqueName: \"kubernetes.io/projected/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-kube-api-access-tf2kv\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.668854 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.668825 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:49.684938 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.684910 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-podinfo\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.685049 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.684986 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-credential-socket\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.685049 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.685023 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcv92\" (UniqueName: \"kubernetes.io/projected/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-kube-api-access-dcv92\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.685049 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.685044 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istiod-ca-cert\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.685245 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.685085 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-envoy\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.685423 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.685375 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-data\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.685534 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.685433 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-credential-socket\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.685534 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.685478 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-workload-socket\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.685534 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.685516 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-token\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.685699 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.685549 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-workload-certs\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.685757 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.685710 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-data\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.685841 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.685815 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-workload-socket\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.685904 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.685841 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istiod-ca-cert\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.686011 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.685977 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-workload-certs\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.687675 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.687654 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-envoy\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.687901 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.687886 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-podinfo\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.709011 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.708981 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-token\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.709266 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.709243 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcv92\" (UniqueName: \"kubernetes.io/projected/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-kube-api-access-dcv92\") pod \"e2e-worker-w6-484607-openshift-default-b8b957995-zbp25\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:49.779622 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.779588 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69"] Oct 30 13:06:49.787536 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.787457 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.808745 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.808456 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69"] Oct 30 13:06:49.891889 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.891806 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5sn5\" (UniqueName: \"kubernetes.io/projected/6e53092c-b84c-406f-badd-976aba85f453-kube-api-access-d5sn5\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.892031 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.891889 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e53092c-b84c-406f-badd-976aba85f453-istio-token\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.892031 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.891961 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-workload-certs\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.892031 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.891977 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-istio-data\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.892167 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.892103 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-istio-envoy\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.892167 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.892135 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e53092c-b84c-406f-badd-976aba85f453-istio-podinfo\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.892233 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.892191 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-workload-socket\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.892285 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.892270 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-credential-socket\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.892354 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.892341 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e53092c-b84c-406f-badd-976aba85f453-istiod-ca-cert\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.990090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.990036 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg"] Oct 30 13:06:49.991985 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:06:49.991960 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f6573b8_e44b_4e54_9b3d_fba69a3dd375.slice/crio-dbd37aeb7777bd3163cd0a678ba5aee766c22a2b921d41a4f290171da2851551 WatchSource:0}: Error finding container dbd37aeb7777bd3163cd0a678ba5aee766c22a2b921d41a4f290171da2851551: Status 404 returned error can't find the container with id dbd37aeb7777bd3163cd0a678ba5aee766c22a2b921d41a4f290171da2851551 Oct 30 13:06:49.993104 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.993079 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e53092c-b84c-406f-badd-976aba85f453-istiod-ca-cert\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.993224 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.993159 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5sn5\" (UniqueName: \"kubernetes.io/projected/6e53092c-b84c-406f-badd-976aba85f453-kube-api-access-d5sn5\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.993224 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.993177 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e53092c-b84c-406f-badd-976aba85f453-istio-token\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.993931 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.993234 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-workload-certs\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.993931 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.993258 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-istio-data\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.993931 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.993299 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-istio-envoy\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.993931 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.993328 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e53092c-b84c-406f-badd-976aba85f453-istio-podinfo\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.993931 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.993377 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-workload-socket\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.993931 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.993432 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-credential-socket\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.993931 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.993818 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e53092c-b84c-406f-badd-976aba85f453-istiod-ca-cert\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.993931 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.993826 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-workload-socket\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.993931 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.993877 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-credential-socket\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.994392 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.993948 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-workload-certs\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.994392 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.994106 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-istio-data\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.995051 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.995019 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:06:49.995177 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.995095 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:06:49.995177 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.995131 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:06:49.995870 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.995852 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-istio-envoy\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:49.996017 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:49.995999 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e53092c-b84c-406f-badd-976aba85f453-istio-podinfo\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:50.005680 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:50.005662 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:50.017544 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:50.017522 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e53092c-b84c-406f-badd-976aba85f453-istio-token\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:50.017628 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:50.017608 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5sn5\" (UniqueName: \"kubernetes.io/projected/6e53092c-b84c-406f-badd-976aba85f453-kube-api-access-d5sn5\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:50.105275 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:50.105245 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:50.328128 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:50.328100 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25"] Oct 30 13:06:50.329695 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:06:50.329668 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ef12288_2a06_4ad5_84ad_df2bcb9331d0.slice/crio-c91004fee529af2613c20f425f0ddd9ba73eb29672f207f2773ab25becee73c6 WatchSource:0}: Error finding container c91004fee529af2613c20f425f0ddd9ba73eb29672f207f2773ab25becee73c6: Status 404 returned error can't find the container with id c91004fee529af2613c20f425f0ddd9ba73eb29672f207f2773ab25becee73c6 Oct 30 13:06:50.332487 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:50.332447 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:06:50.332588 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:50.332512 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:06:50.332588 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:50.332543 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:06:50.428683 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:50.428655 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69"] Oct 30 13:06:50.431243 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:06:50.431213 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e53092c_b84c_406f_badd_976aba85f453.slice/crio-8d27bb19b0ffb05c530f3b74e6a05b853b19760d0fdbb31cc4571e48b199ba2c WatchSource:0}: Error finding container 8d27bb19b0ffb05c530f3b74e6a05b853b19760d0fdbb31cc4571e48b199ba2c: Status 404 returned error can't find the container with id 8d27bb19b0ffb05c530f3b74e6a05b853b19760d0fdbb31cc4571e48b199ba2c Oct 30 13:06:50.434626 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:50.434584 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:06:50.434737 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:50.434671 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:06:50.434737 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:50.434714 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:06:50.530245 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:50.529312 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" event={"ID":"6e53092c-b84c-406f-badd-976aba85f453","Type":"ContainerStarted","Data":"b05607f2471236a1a04b70cdfd4fd1447766daefbf1e4b0fe7fdc2becb28ee30"} Oct 30 13:06:50.530245 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:50.529382 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" event={"ID":"6e53092c-b84c-406f-badd-976aba85f453","Type":"ContainerStarted","Data":"8d27bb19b0ffb05c530f3b74e6a05b853b19760d0fdbb31cc4571e48b199ba2c"} Oct 30 13:06:50.533225 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:50.532578 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" event={"ID":"6ef12288-2a06-4ad5-84ad-df2bcb9331d0","Type":"ContainerStarted","Data":"16a98e96090107e7caefd3a40cc770cedda99f43c802cd9c4a504b467223d92c"} Oct 30 13:06:50.533225 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:50.532633 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" event={"ID":"6ef12288-2a06-4ad5-84ad-df2bcb9331d0","Type":"ContainerStarted","Data":"c91004fee529af2613c20f425f0ddd9ba73eb29672f207f2773ab25becee73c6"} Oct 30 13:06:50.535858 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:50.535593 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" event={"ID":"4f6573b8-e44b-4e54-9b3d-fba69a3dd375","Type":"ContainerStarted","Data":"37bcba6fec8dfe8eddeeafd8c6366d9f8b0dcec2e3e935bbcb8af7bd30a9ecb8"} Oct 30 13:06:50.535858 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:50.535627 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" event={"ID":"4f6573b8-e44b-4e54-9b3d-fba69a3dd375","Type":"ContainerStarted","Data":"dbd37aeb7777bd3163cd0a678ba5aee766c22a2b921d41a4f290171da2851551"} Oct 30 13:06:50.580407 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:50.580347 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" podStartSLOduration=1.580326889 podStartE2EDuration="1.580326889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:06:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:06:50.57461219 +0000 UTC m=+909.600253463" watchObservedRunningTime="2026-10-30 13:06:50.580326889 +0000 UTC m=+909.605968143" Oct 30 13:06:50.621853 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:50.621788 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" podStartSLOduration=1.621769611 podStartE2EDuration="1.621769611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:06:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:06:50.618962258 +0000 UTC m=+909.644603497" watchObservedRunningTime="2026-10-30 13:06:50.621769611 +0000 UTC m=+909.647410861" Oct 30 13:06:50.668368 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:50.668306 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" podStartSLOduration=1.668284956 podStartE2EDuration="1.668284956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:06:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:06:50.662922012 +0000 UTC m=+909.688563287" watchObservedRunningTime="2026-10-30 13:06:50.668284956 +0000 UTC m=+909.693926207" Oct 30 13:06:50.669420 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:50.669396 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:06:51.005862 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:51.005824 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:06:51.007628 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:51.007601 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Oct 30 13:06:51.007737 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:51.007652 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" podUID="6ef12288-2a06-4ad5-84ad-df2bcb9331d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Oct 30 13:06:51.105495 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:51.105466 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:51.106846 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:51.106817 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Oct 30 13:06:51.106976 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:51.106863 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" podUID="6e53092c-b84c-406f-badd-976aba85f453" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Oct 30 13:06:51.682092 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:51.678224 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 13:06:51.682092 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:51.678287 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" podUID="4f6573b8-e44b-4e54-9b3d-fba69a3dd375" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 13:06:52.010060 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:52.009964 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Oct 30 13:06:52.010363 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:52.010329 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" podUID="6ef12288-2a06-4ad5-84ad-df2bcb9331d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Oct 30 13:06:52.112528 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:52.112486 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Oct 30 13:06:52.112805 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:52.112776 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" podUID="6e53092c-b84c-406f-badd-976aba85f453" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Oct 30 13:06:52.669794 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:52.669763 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 13:06:52.669981 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:52.669812 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" podUID="4f6573b8-e44b-4e54-9b3d-fba69a3dd375" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 13:06:53.031148 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:53.030473 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Oct 30 13:06:53.031148 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:53.030534 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" podUID="6ef12288-2a06-4ad5-84ad-df2bcb9331d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Oct 30 13:06:53.148091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:53.146165 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Oct 30 13:06:53.148091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:53.146224 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" podUID="6e53092c-b84c-406f-badd-976aba85f453" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Oct 30 13:06:53.669250 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:53.669218 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 13:06:53.669482 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:53.669461 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" podUID="4f6573b8-e44b-4e54-9b3d-fba69a3dd375" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 13:06:54.007143 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:54.006540 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Oct 30 13:06:54.007143 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:54.006606 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" podUID="6ef12288-2a06-4ad5-84ad-df2bcb9331d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Oct 30 13:06:54.107091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:54.106301 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Oct 30 13:06:54.107091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:54.106364 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" podUID="6e53092c-b84c-406f-badd-976aba85f453" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Oct 30 13:06:54.669278 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:54.669239 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 13:06:54.669482 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:54.669303 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" podUID="4f6573b8-e44b-4e54-9b3d-fba69a3dd375" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 13:06:55.014557 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:55.014462 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Oct 30 13:06:55.014819 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:55.014791 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" podUID="6ef12288-2a06-4ad5-84ad-df2bcb9331d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Oct 30 13:06:55.124090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:55.106530 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Oct 30 13:06:55.124090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:55.106593 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" podUID="6e53092c-b84c-406f-badd-976aba85f453" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Oct 30 13:06:55.669626 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:55.669593 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 13:06:55.669797 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:55.669642 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" podUID="4f6573b8-e44b-4e54-9b3d-fba69a3dd375" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 13:06:56.008597 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:56.008502 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Oct 30 13:06:56.008852 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:56.008824 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" podUID="6ef12288-2a06-4ad5-84ad-df2bcb9331d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Oct 30 13:06:56.108512 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:56.108468 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Oct 30 13:06:56.108794 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:56.108762 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" podUID="6e53092c-b84c-406f-badd-976aba85f453" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Oct 30 13:06:56.669462 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:56.669424 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 13:06:56.669999 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:56.669471 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" podUID="4f6573b8-e44b-4e54-9b3d-fba69a3dd375" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 13:06:57.008055 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:57.007351 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Oct 30 13:06:57.008055 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:57.007426 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" podUID="6ef12288-2a06-4ad5-84ad-df2bcb9331d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Oct 30 13:06:57.106987 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:57.106462 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Oct 30 13:06:57.106987 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:57.106521 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" podUID="6e53092c-b84c-406f-badd-976aba85f453" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Oct 30 13:06:57.675202 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:57.675164 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 13:06:57.675779 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:57.675747 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" podUID="4f6573b8-e44b-4e54-9b3d-fba69a3dd375" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 13:06:58.029145 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:58.011565 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Oct 30 13:06:58.029145 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:58.011655 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" podUID="6ef12288-2a06-4ad5-84ad-df2bcb9331d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Oct 30 13:06:58.112092 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:58.108499 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Oct 30 13:06:58.112092 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:58.108576 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" podUID="6e53092c-b84c-406f-badd-976aba85f453" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Oct 30 13:06:59.669524 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:59.669483 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 13:06:59.670157 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:59.670124 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" podUID="4f6573b8-e44b-4e54-9b3d-fba69a3dd375" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 13:06:59.782704 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:59.782670 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:59.784100 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:59.784043 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:06:59.793710 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:06:59.789692 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:07:00.030144 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:00.023220 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 13:07:00.030144 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:00.023285 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" podUID="6ef12288-2a06-4ad5-84ad-df2bcb9331d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 13:07:00.718090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:00.717155 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 13:07:00.718090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:00.717218 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" podUID="4f6573b8-e44b-4e54-9b3d-fba69a3dd375" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 13:07:00.896088 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:00.891799 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:07:00.896088 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:00.893111 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:07:00.896333 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:00.896265 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:07:01.066253 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:01.066192 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 13:07:01.066447 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:01.066260 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" podUID="6ef12288-2a06-4ad5-84ad-df2bcb9331d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 13:07:01.363962 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:01.363882 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:07:01.364353 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:01.364328 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:07:01.365555 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:01.365534 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:07:14.879210 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:07:14.879174 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:07:19.881317 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:19.881283 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw"] Oct 30 13:07:19.885464 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:19.885444 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:19.907233 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:19.907200 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-token\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:19.907449 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:19.907424 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-podinfo\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:19.907539 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:19.907461 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/499f565c-be4d-4573-ab85-d78f68bf43b7-istiod-ca-cert\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:19.907539 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:19.907508 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-envoy\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:19.907696 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:19.907648 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-data\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:19.907696 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:19.907679 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-workload-certs\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:19.907801 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:19.907713 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mj9hw\" (UniqueName: \"kubernetes.io/projected/499f565c-be4d-4573-ab85-d78f68bf43b7-kube-api-access-mj9hw\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:19.907801 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:19.907744 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-credential-socket\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:19.907921 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:19.907831 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-workload-socket\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:19.910532 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:19.910498 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw"] Oct 30 13:07:20.009172 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.009138 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-token\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:20.009342 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.009187 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-podinfo\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:20.009342 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.009229 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/499f565c-be4d-4573-ab85-d78f68bf43b7-istiod-ca-cert\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:20.009342 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.009276 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-envoy\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:20.009523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.009370 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-data\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:20.009523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.009386 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-workload-certs\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:20.009523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.009409 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mj9hw\" (UniqueName: \"kubernetes.io/projected/499f565c-be4d-4573-ab85-d78f68bf43b7-kube-api-access-mj9hw\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:20.009523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.009427 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-credential-socket\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:20.009523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.009511 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-workload-socket\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:20.009878 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.009851 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-data\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:20.009960 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.009914 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-workload-socket\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:20.010019 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.009962 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-credential-socket\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:20.010104 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.010019 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/499f565c-be4d-4573-ab85-d78f68bf43b7-istiod-ca-cert\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:20.010104 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.010030 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-workload-certs\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:20.011681 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.011659 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-envoy\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:20.011824 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.011808 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-podinfo\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:20.033482 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.033454 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-token\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:20.034359 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.034340 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mj9hw\" (UniqueName: \"kubernetes.io/projected/499f565c-be4d-4573-ab85-d78f68bf43b7-kube-api-access-mj9hw\") pod \"e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:20.199987 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.199909 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:20.333248 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.333212 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9"] Oct 30 13:07:20.338033 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.338009 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.392397 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.392362 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9"] Oct 30 13:07:20.415647 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.415618 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-envoy\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.415795 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.415655 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-workload-certs\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.415795 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.415680 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-podinfo\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.415890 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.415869 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-token\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.415926 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.415915 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-data\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.415964 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.415937 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh8vx\" (UniqueName: \"kubernetes.io/projected/c461dc10-21d9-40ea-ad0b-1125e530b240-kube-api-access-gh8vx\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.416008 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.415977 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-credential-socket\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.416008 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.415994 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c461dc10-21d9-40ea-ad0b-1125e530b240-istiod-ca-cert\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.416244 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.416151 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-workload-socket\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.430553 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.430527 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb"] Oct 30 13:07:20.436791 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.436771 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.484009 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.483916 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb"] Oct 30 13:07:20.517393 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.517361 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-credential-socket\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.517559 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.517436 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb32630c-51c4-495d-8a43-eed34246ada8-istiod-ca-cert\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.517559 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.517536 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-token\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.517646 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.517590 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjrnx\" (UniqueName: \"kubernetes.io/projected/eb32630c-51c4-495d-8a43-eed34246ada8-kube-api-access-jjrnx\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.517646 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.517625 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb32630c-51c4-495d-8a43-eed34246ada8-istio-token\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.517742 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.517668 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-data\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.517742 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.517694 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh8vx\" (UniqueName: \"kubernetes.io/projected/c461dc10-21d9-40ea-ad0b-1125e530b240-kube-api-access-gh8vx\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.517807 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.517773 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-credential-socket\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.517843 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.517809 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c461dc10-21d9-40ea-ad0b-1125e530b240-istiod-ca-cert\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.517885 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.517837 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-workload-socket\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.517937 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.517919 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb32630c-51c4-495d-8a43-eed34246ada8-istio-podinfo\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.518053 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.518035 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-workload-socket\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.518147 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.518132 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-workload-certs\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.518208 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.518166 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-istio-envoy\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.518263 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.518235 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-envoy\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.518323 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.518256 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-data\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.518323 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.518271 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-workload-certs\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.518424 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.518333 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-podinfo\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.518424 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.518388 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-istio-data\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.518424 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.518417 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-credential-socket\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.518803 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.518780 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-workload-socket\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.518967 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.518826 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-workload-certs\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.519179 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.519156 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c461dc10-21d9-40ea-ad0b-1125e530b240-istiod-ca-cert\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.520593 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.520567 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-envoy\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.521036 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.521014 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-podinfo\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.541512 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.541492 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh8vx\" (UniqueName: \"kubernetes.io/projected/c461dc10-21d9-40ea-ad0b-1125e530b240-kube-api-access-gh8vx\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.542331 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.542310 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-token\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.553883 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.553861 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw"] Oct 30 13:07:20.556255 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:07:20.556220 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod499f565c_be4d_4573_ab85_d78f68bf43b7.slice/crio-54aa88c734bb42914d36abb2679c0412b99c5dea8474492c921fbe7c98574d07 WatchSource:0}: Error finding container 54aa88c734bb42914d36abb2679c0412b99c5dea8474492c921fbe7c98574d07: Status 404 returned error can't find the container with id 54aa88c734bb42914d36abb2679c0412b99c5dea8474492c921fbe7c98574d07 Oct 30 13:07:20.558837 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.558804 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:07:20.558931 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.558867 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:07:20.558931 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.558894 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:07:20.619690 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.619664 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-workload-certs\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.619761 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.619711 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-istio-envoy\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.619812 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.619797 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-istio-data\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.619875 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.619859 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-credential-socket\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.619941 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.619928 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb32630c-51c4-495d-8a43-eed34246ada8-istiod-ca-cert\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.619996 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.619985 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjrnx\" (UniqueName: \"kubernetes.io/projected/eb32630c-51c4-495d-8a43-eed34246ada8-kube-api-access-jjrnx\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.620054 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.620016 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb32630c-51c4-495d-8a43-eed34246ada8-istio-token\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.620054 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.620043 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-workload-certs\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.620205 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.620185 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-workload-socket\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.620267 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.620243 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-istio-data\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.620267 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.620259 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-credential-socket\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.620372 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.620261 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb32630c-51c4-495d-8a43-eed34246ada8-istio-podinfo\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.620616 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.620592 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-workload-socket\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.621013 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.620992 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb32630c-51c4-495d-8a43-eed34246ada8-istiod-ca-cert\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.622165 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.622145 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-istio-envoy\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.622485 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.622465 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb32630c-51c4-495d-8a43-eed34246ada8-istio-podinfo\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.650705 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.650666 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjrnx\" (UniqueName: \"kubernetes.io/projected/eb32630c-51c4-495d-8a43-eed34246ada8-kube-api-access-jjrnx\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.653357 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.653334 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:20.672772 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.672749 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb32630c-51c4-495d-8a43-eed34246ada8-istio-token\") pod \"e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.751897 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.751808 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:20.888976 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.888937 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" event={"ID":"499f565c-be4d-4573-ab85-d78f68bf43b7","Type":"ContainerStarted","Data":"ac5ddf68af04694bfdc3f310b914ed2f451daa3748bd3cb5853a8a630cdbf723"} Oct 30 13:07:20.889500 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.888999 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" event={"ID":"499f565c-be4d-4573-ab85-d78f68bf43b7","Type":"ContainerStarted","Data":"54aa88c734bb42914d36abb2679c0412b99c5dea8474492c921fbe7c98574d07"} Oct 30 13:07:20.938489 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.938434 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" podStartSLOduration=1.938418832 podStartE2EDuration="1.938418832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:07:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:07:20.935919445 +0000 UTC m=+939.961560704" watchObservedRunningTime="2026-10-30 13:07:20.938418832 +0000 UTC m=+939.964060080" Oct 30 13:07:20.985510 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.985484 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9"] Oct 30 13:07:20.992588 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.992555 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:07:20.992691 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.992622 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:07:20.992691 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:20.992659 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:07:21.074904 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:21.074870 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb"] Oct 30 13:07:21.090963 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:21.090924 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:07:21.091191 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:21.091166 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:07:21.091336 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:21.091317 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:07:21.201174 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:21.201082 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:21.205366 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:21.205214 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 30 13:07:21.205366 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:21.205274 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" podUID="499f565c-be4d-4573-ab85-d78f68bf43b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 30 13:07:21.916989 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:21.916917 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" event={"ID":"eb32630c-51c4-495d-8a43-eed34246ada8","Type":"ContainerStarted","Data":"c142c44181093d993c3ab7115ed95392d6a049e2e7a2c9f2bbb18a7bfd434e31"} Oct 30 13:07:21.917617 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:21.917591 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" event={"ID":"eb32630c-51c4-495d-8a43-eed34246ada8","Type":"ContainerStarted","Data":"325648bded87fabfadead5f4ad8b67e1b415bd905d09dafe139c6e7f3c3cdeb0"} Oct 30 13:07:21.938102 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:21.938050 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" event={"ID":"c461dc10-21d9-40ea-ad0b-1125e530b240","Type":"ContainerStarted","Data":"b3c0da523bd1cd996a813f7fc382f288183417a243dfd0717ce49827050b653c"} Oct 30 13:07:21.946340 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:21.946304 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" event={"ID":"c461dc10-21d9-40ea-ad0b-1125e530b240","Type":"ContainerStarted","Data":"bbded2b80d08790493d70f78419a43166dfc62c6ad72eda899ef6cf45f2cbc14"} Oct 30 13:07:22.053169 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:22.051013 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" podStartSLOduration=2.050992296 podStartE2EDuration="2.050992296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:07:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:07:21.990420048 +0000 UTC m=+941.016061326" watchObservedRunningTime="2026-10-30 13:07:22.050992296 +0000 UTC m=+941.076633546" Oct 30 13:07:22.053169 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:22.051423 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" podStartSLOduration=2.051410107 podStartE2EDuration="2.051410107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:07:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:07:22.04852864 +0000 UTC m=+941.074169910" watchObservedRunningTime="2026-10-30 13:07:22.051410107 +0000 UTC m=+941.077051356" Oct 30 13:07:22.211148 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:22.207575 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 30 13:07:22.211148 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:22.207637 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" podUID="499f565c-be4d-4573-ab85-d78f68bf43b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 30 13:07:22.663384 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:22.661983 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:22.663802 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:22.663747 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 30 13:07:22.663887 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:22.663800 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" podUID="c461dc10-21d9-40ea-ad0b-1125e530b240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 30 13:07:22.761133 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:22.759565 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:22.763091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:22.761977 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 30 13:07:22.763091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:22.762056 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" podUID="eb32630c-51c4-495d-8a43-eed34246ada8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 30 13:07:23.217098 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:23.216186 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 30 13:07:23.217098 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:23.216255 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" podUID="499f565c-be4d-4573-ab85-d78f68bf43b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 30 13:07:23.654219 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:23.654181 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 30 13:07:23.654481 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:23.654456 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" podUID="c461dc10-21d9-40ea-ad0b-1125e530b240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 30 13:07:23.752627 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:23.752595 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 30 13:07:23.752871 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:23.752844 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" podUID="eb32630c-51c4-495d-8a43-eed34246ada8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 30 13:07:24.258605 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:24.258339 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 30 13:07:24.258605 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:24.258400 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" podUID="499f565c-be4d-4573-ab85-d78f68bf43b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 30 13:07:24.655218 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:24.654037 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 30 13:07:24.655218 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:24.654109 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" podUID="c461dc10-21d9-40ea-ad0b-1125e530b240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 30 13:07:24.753090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:24.752737 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 30 13:07:24.753090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:24.752797 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" podUID="eb32630c-51c4-495d-8a43-eed34246ada8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 30 13:07:25.219092 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:25.203387 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 30 13:07:25.219092 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:25.203447 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" podUID="499f565c-be4d-4573-ab85-d78f68bf43b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 30 13:07:25.656274 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:25.654180 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 30 13:07:25.656274 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:25.654257 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" podUID="c461dc10-21d9-40ea-ad0b-1125e530b240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 30 13:07:25.753557 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:25.753519 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 30 13:07:25.753845 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:25.753813 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" podUID="eb32630c-51c4-495d-8a43-eed34246ada8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 30 13:07:26.207089 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:26.206090 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 30 13:07:26.207089 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:26.206246 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" podUID="499f565c-be4d-4573-ab85-d78f68bf43b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 30 13:07:26.654438 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:26.654398 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 30 13:07:26.654617 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:26.654470 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" podUID="c461dc10-21d9-40ea-ad0b-1125e530b240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 30 13:07:26.753490 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:26.753443 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 30 13:07:26.753955 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:26.753509 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" podUID="eb32630c-51c4-495d-8a43-eed34246ada8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 30 13:07:27.200425 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:27.200386 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 30 13:07:27.200690 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:27.200662 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" podUID="499f565c-be4d-4573-ab85-d78f68bf43b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 30 13:07:27.656153 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:27.656109 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 30 13:07:27.656339 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:27.656180 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" podUID="c461dc10-21d9-40ea-ad0b-1125e530b240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 30 13:07:27.752532 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:27.752501 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 30 13:07:27.752726 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:27.752695 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" podUID="eb32630c-51c4-495d-8a43-eed34246ada8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 30 13:07:28.424802 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:28.404519 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:28.424802 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:28.406105 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:28.424802 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:28.407499 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:07:28.977286 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:28.977252 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:28.978703 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:28.978680 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:28.979535 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:28.979513 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:07:29.025888 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:29.025835 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:29.026302 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:29.026273 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:29.028553 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:29.028528 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:07:34.333505 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:07:34.333453 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" podUID="19409b77-526f-48f7-b539-6d8ebb2d3dfc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 13:07:44.884467 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:07:44.884417 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:08:05.772438 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:05.772386 5029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" podUID="b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:06.713093 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:06.711172 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" podUID="2b878b27-465c-433d-aba4-e31849cfc14c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:07.204242 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:07.204195 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" podUID="b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:07.329213 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:07.329156 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" podUID="19409b77-526f-48f7-b539-6d8ebb2d3dfc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:07.395091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:07.381422 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" podUID="99e695e7-e73c-429c-87ca-39ee740917c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded" Oct 30 13:08:07.440233 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:07.440181 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" podUID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded" Oct 30 13:08:07.585151 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:07.575482 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" podUID="b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:14.879332 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:08:14.879295 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:08:21.211199 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:21.211149 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" podUID="b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded" Oct 30 13:08:21.294213 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:21.294162 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" podUID="d6bdcd3f-0200-48e6-af85-2d1d8f022054" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:21.352235 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:21.352184 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" podUID="19409b77-526f-48f7-b539-6d8ebb2d3dfc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:21.363180 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:21.363142 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" podUID="99e695e7-e73c-429c-87ca-39ee740917c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:21.423990 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:21.423946 5029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" podUID="6962e138-faa9-438c-9138-1133bdce1011" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:21.433389 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:21.433344 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" podUID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:21.529521 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:21.529398 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" podUID="6962e138-faa9-438c-9138-1133bdce1011" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:21.559194 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:21.558484 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" podUID="b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:21.645215 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:21.645167 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" podUID="2b878b27-465c-433d-aba4-e31849cfc14c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:22.215423 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:22.213163 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" podUID="b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:22.215423 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:22.213919 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:08:22.470149 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:22.457127 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" podUID="99e695e7-e73c-429c-87ca-39ee740917c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:22.534091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:22.471110 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:08:22.638794 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:22.565464 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" podUID="b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:22.638794 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:22.638725 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:08:22.656313 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:22.656232 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" podUID="2b878b27-465c-433d-aba4-e31849cfc14c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded" Oct 30 13:08:22.657589 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:22.656734 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:08:23.251357 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:23.249830 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" podUID="b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded" Oct 30 13:08:23.251357 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:23.251010 5029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" podUID="2b878b27-465c-433d-aba4-e31849cfc14c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:23.301367 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:23.301317 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" podUID="d6bdcd3f-0200-48e6-af85-2d1d8f022054" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded" Oct 30 13:08:23.333691 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:23.333655 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" podUID="19409b77-526f-48f7-b539-6d8ebb2d3dfc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:23.363275 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:23.363244 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:08:23.373203 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:23.373176 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:08:23.500089 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:23.492233 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" podUID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:23.504136 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:23.494139 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" podUID="99e695e7-e73c-429c-87ca-39ee740917c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:23.525194 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:23.525152 5029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" podUID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:23.568450 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:23.568406 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" podUID="6962e138-faa9-438c-9138-1133bdce1011" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded" Oct 30 13:08:23.743843 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:23.743812 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:08:23.876093 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:23.875186 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:08:41.301913 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:41.301176 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" podUID="d6bdcd3f-0200-48e6-af85-2d1d8f022054" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:41.337668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:41.337536 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" podUID="19409b77-526f-48f7-b539-6d8ebb2d3dfc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded" Oct 30 13:08:41.426257 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:41.426205 5029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" podUID="6962e138-faa9-438c-9138-1133bdce1011" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:41.429378 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:41.429343 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" podUID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:41.534222 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:41.534167 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" podUID="6962e138-faa9-438c-9138-1133bdce1011" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:41.558145 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:41.537319 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" podUID="99e695e7-e73c-429c-87ca-39ee740917c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded" Oct 30 13:08:41.747219 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:41.747172 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" podUID="b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded" Oct 30 13:08:42.128172 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:42.125170 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" podUID="b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:42.128172 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:42.125599 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" podUID="2b878b27-465c-433d-aba4-e31849cfc14c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:42.307196 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:42.307149 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" podUID="d6bdcd3f-0200-48e6-af85-2d1d8f022054" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded" Oct 30 13:08:42.308486 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:42.308462 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:08:42.555154 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:42.543828 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" podUID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded" Oct 30 13:08:42.588091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:42.560144 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" podUID="6962e138-faa9-438c-9138-1133bdce1011" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded" Oct 30 13:08:42.588658 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:42.572780 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:08:42.588911 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:42.588894 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:08:42.604089 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:42.575273 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" podUID="99e695e7-e73c-429c-87ca-39ee740917c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:42.640088 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:42.640029 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:08:42.768309 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:42.753328 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" podUID="b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:42.768309 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:42.754031 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:08:42.939108 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:42.934183 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:08:43.145689 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:43.145646 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" podUID="b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded" Oct 30 13:08:43.169313 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:43.169266 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" podUID="2b878b27-465c-433d-aba4-e31849cfc14c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:43.257341 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:43.257237 5029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" podUID="2b878b27-465c-433d-aba4-e31849cfc14c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:43.263562 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:43.263527 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:08:43.268648 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:43.268629 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:08:43.268859 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:43.268848 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:08:43.315585 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:43.315535 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" podUID="d6bdcd3f-0200-48e6-af85-2d1d8f022054" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:43.330846 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:43.330795 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" podUID="19409b77-526f-48f7-b539-6d8ebb2d3dfc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:43.373309 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:43.373285 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:08:43.382677 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:43.382653 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:08:43.569123 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:43.568175 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" podUID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:43.577753 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:43.577714 5029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" podUID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/health\": context deadline exceeded" Oct 30 13:08:43.623088 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:43.619561 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 13:08:43.666291 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:43.656125 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 13:08:43.666291 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:43.666227 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" podUID="88f14514-918e-43a1-a097-c236701d22f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:43.666793 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:43.666768 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" podUID="17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:44.095090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:44.078280 5029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" podUID="19409b77-526f-48f7-b539-6d8ebb2d3dfc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/health\": context deadline exceeded" Oct 30 13:08:44.146664 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:44.146571 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" podUID="b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded" Oct 30 13:08:44.232220 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:44.232175 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 13:08:44.232504 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:44.232469 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" podUID="184f5837-5aee-46b3-856c-4c3db17a7877" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:44.234721 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:44.234687 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" podUID="2b878b27-465c-433d-aba4-e31849cfc14c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded" Oct 30 13:08:44.280101 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:44.258650 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" podUID="99e695e7-e73c-429c-87ca-39ee740917c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:44.335200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:44.335155 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" podUID="19409b77-526f-48f7-b539-6d8ebb2d3dfc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:44.343203 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:44.343172 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:08:44.492255 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:44.492154 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 13:08:44.492255 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:44.492224 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" podUID="499f565c-be4d-4573-ab85-d78f68bf43b7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:44.494143 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:44.494113 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 13:08:44.494245 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:44.494164 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" podUID="48dfe4a1-9d55-4f04-812a-1d0084f01c7e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:44.600972 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:44.600925 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" podUID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:44.793148 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:44.757633 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:08:44.793148 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:44.757702 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:08:44.849317 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:44.846641 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:08:44.961192 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:44.960470 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" podUID="b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:45.059610 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:45.059209 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 13:08:45.059610 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:45.059271 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" podUID="c461dc10-21d9-40ea-ad0b-1125e530b240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:45.127330 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:45.127148 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 13:08:45.127330 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:45.127211 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" podUID="eb32630c-51c4-495d-8a43-eed34246ada8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:45.324252 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:45.324204 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" podUID="d6bdcd3f-0200-48e6-af85-2d1d8f022054" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:45.446089 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:45.442722 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:08:45.471189 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:08:45.467351 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:08:45.609558 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:45.609209 5029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" podUID="d6bdcd3f-0200-48e6-af85-2d1d8f022054" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:08:45.908252 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:45.908154 5029 patch_prober.go:29] interesting pod/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 13:08:45.926107 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:08:45.911522 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" podUID="6e53092c-b84c-406f-badd-976aba85f453" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:05.577260 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:05.576587 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5"] Oct 30 13:09:05.732089 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:05.731418 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:05.899489 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:05.834436 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5"] Oct 30 13:09:05.899489 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:05.852173 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" podUID="b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:05.930280 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:05.930243 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-istio-data\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:05.930697 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:05.930668 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-istio-envoy\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:05.934193 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:05.934147 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/617de128-a9b7-428c-acec-3753566fba40-istio-token\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:05.934398 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:05.934377 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-workload-certs\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:05.938147 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:05.938121 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/617de128-a9b7-428c-acec-3753566fba40-istiod-ca-cert\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:05.938317 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:05.938297 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvg2n\" (UniqueName: \"kubernetes.io/projected/617de128-a9b7-428c-acec-3753566fba40-kube-api-access-hvg2n\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:05.943211 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:05.943184 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/617de128-a9b7-428c-acec-3753566fba40-istio-podinfo\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:05.943461 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:05.943440 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-credential-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:05.947234 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:05.947209 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-workload-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:06.060133 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.060086 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-istio-envoy\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:06.060330 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.060151 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/617de128-a9b7-428c-acec-3753566fba40-istio-token\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:06.060330 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.060195 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-workload-certs\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:06.060330 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.060223 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/617de128-a9b7-428c-acec-3753566fba40-istiod-ca-cert\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:06.060330 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.060256 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvg2n\" (UniqueName: \"kubernetes.io/projected/617de128-a9b7-428c-acec-3753566fba40-kube-api-access-hvg2n\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:06.060330 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.060320 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/617de128-a9b7-428c-acec-3753566fba40-istio-podinfo\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:06.060604 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.060398 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-credential-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:06.060604 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.060469 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-workload-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:06.060604 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.060510 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-istio-data\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:06.072107 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.061002 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-istio-data\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:06.072107 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.068858 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-workload-certs\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:06.072107 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.069552 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/617de128-a9b7-428c-acec-3753566fba40-istiod-ca-cert\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:06.087130 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.080799 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/617de128-a9b7-428c-acec-3753566fba40-istio-podinfo\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:06.087130 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.081090 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-workload-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:06.087130 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.081342 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-credential-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:06.087130 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.081514 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-istio-envoy\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:06.098909 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.098878 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvg2n\" (UniqueName: \"kubernetes.io/projected/617de128-a9b7-428c-acec-3753566fba40-kube-api-access-hvg2n\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:06.105669 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.105639 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/617de128-a9b7-428c-acec-3753566fba40-istio-token\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:06.163144 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.162292 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks"] Oct 30 13:09:06.209092 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.204384 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" podUID="b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded" Oct 30 13:09:06.291220 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.272994 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:06.324208 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.324149 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" podUID="d6bdcd3f-0200-48e6-af85-2d1d8f022054" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded" Oct 30 13:09:06.387737 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.386265 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks"] Oct 30 13:09:06.387737 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.386390 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.649090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.631687 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" podUID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:06.676091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.631745 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-podinfo\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.691090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.676356 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-workload-certs\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.699257 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.699222 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-token\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.699536 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.699516 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-workload-socket\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.699652 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.699636 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-credential-socket\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.699750 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.699737 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-data\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.699856 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.699843 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mzsf\" (UniqueName: \"kubernetes.io/projected/efa3fa51-d5f3-49c2-9658-378b1a6a2719-kube-api-access-5mzsf\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.699970 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.699958 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-envoy\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.700105 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.700090 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istiod-ca-cert\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.760142 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.759208 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" podUID="2b878b27-465c-433d-aba4-e31849cfc14c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:06.806906 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.805867 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istiod-ca-cert\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.806906 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.806058 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-podinfo\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.806906 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.806132 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-workload-certs\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.806906 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.806214 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-token\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.806906 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.806284 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-workload-socket\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.806906 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.806310 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-credential-socket\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.806906 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.806332 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-data\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.806906 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.806362 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mzsf\" (UniqueName: \"kubernetes.io/projected/efa3fa51-d5f3-49c2-9658-378b1a6a2719-kube-api-access-5mzsf\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.806906 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.806397 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-envoy\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.808308 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.808283 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-workload-socket\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.808721 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.808696 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-workload-certs\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.810382 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.810359 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-data\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.812114 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.811142 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istiod-ca-cert\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.816127 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.811747 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-credential-socket\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.851874 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.851791 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-podinfo\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.858260 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.858216 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" podUID="19409b77-526f-48f7-b539-6d8ebb2d3dfc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:06.861031 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.858880 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" podUID="b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded" Oct 30 13:09:06.861031 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.859542 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:09:06.866626 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.866561 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-envoy\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.905106 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.901447 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-token\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:06.911162 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:06.906789 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mzsf\" (UniqueName: \"kubernetes.io/projected/efa3fa51-d5f3-49c2-9658-378b1a6a2719-kube-api-access-5mzsf\") pod \"e2e-worker-w4-54994c-openshift-default-c8547749c-562ks\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:07.169210 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:07.169177 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:07.211439 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:07.210360 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" podUID="b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded" Oct 30 13:09:07.302514 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:07.302453 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" podUID="99e695e7-e73c-429c-87ca-39ee740917c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:07.384583 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:07.384538 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" podUID="d6bdcd3f-0200-48e6-af85-2d1d8f022054" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:07.445899 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:07.445798 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" podUID="6962e138-faa9-438c-9138-1133bdce1011" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:07.660472 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:07.660238 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:09:07.808333 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:07.808242 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:09:07.824092 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:07.792334 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" podUID="2b878b27-465c-433d-aba4-e31849cfc14c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:07.828087 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:07.824788 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:09:07.883299 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:07.883253 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" podUID="b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded" Oct 30 13:09:07.883914 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:07.883884 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" podUID="19409b77-526f-48f7-b539-6d8ebb2d3dfc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded" Oct 30 13:09:07.884453 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:07.884433 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:09:08.218928 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:08.218880 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" podUID="b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:08.307122 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:08.303757 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" podUID="99e695e7-e73c-429c-87ca-39ee740917c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded" Oct 30 13:09:08.307122 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:08.304700 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:09:08.443398 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:08.385720 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" podUID="d6bdcd3f-0200-48e6-af85-2d1d8f022054" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:08.459091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:08.452552 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" podUID="6962e138-faa9-438c-9138-1133bdce1011" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:08.459091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:08.452789 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:09:08.588059 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:08.588012 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" podUID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:08.636231 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:08.636204 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:09:08.837331 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:08.836309 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" podUID="2b878b27-465c-433d-aba4-e31849cfc14c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:08.899369 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:08.898194 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" podUID="19409b77-526f-48f7-b539-6d8ebb2d3dfc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:08.932314 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:08.931692 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" podUID="b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:09.061344 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.061163 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:09:09.100123 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.098981 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:09:09.100123 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.099082 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:09:09.100123 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.099150 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:09:09.200106 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.197197 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:09:09.232505 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.231019 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" podUID="b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded" Oct 30 13:09:09.356124 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.349843 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" podUID="99e695e7-e73c-429c-87ca-39ee740917c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded" Oct 30 13:09:09.487350 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.487315 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks"] Oct 30 13:09:09.550259 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.550200 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" event={"ID":"efa3fa51-d5f3-49c2-9658-378b1a6a2719","Type":"ContainerStarted","Data":"834ccef10eed644f050f3d94d0542555d88058527d4f9b5c359687c41f1a1882"} Oct 30 13:09:09.634137 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.613388 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" podUID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:09.650265 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.634391 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf"] Oct 30 13:09:09.750701 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.739293 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:09:09.750701 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.740234 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:09.787250 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.778848 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:09:09.787250 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.778949 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:09:09.787250 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.779025 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:09:09.787250 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.780481 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-3048f9\"/\"openshift-service-ca.crt\"" Oct 30 13:09:09.787250 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.780584 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-3048f9\"/\"kube-root-ca.crt\"" Oct 30 13:09:09.787250 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.780674 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-3048f9\"/\"default-dockercfg-f7hxq\"" Oct 30 13:09:09.787250 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.780760 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-3048f9\"/\"distinct-w3-3048f9-kserve-self-signed-certs\"" Oct 30 13:09:09.787250 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.781704 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:09:09.866545 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.824810 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf"] Oct 30 13:09:09.866545 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.840534 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" podUID="2b878b27-465c-433d-aba4-e31849cfc14c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded" Oct 30 13:09:09.868185 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.866627 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-dshm\") pod \"distinct-w3-3048f9-kserve-5c85898b56-d6xrf\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:09.868185 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.866708 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pfr8\" (UniqueName: \"kubernetes.io/projected/935c23b1-953a-40cc-ad0c-bec2ef61afba-kube-api-access-9pfr8\") pod \"distinct-w3-3048f9-kserve-5c85898b56-d6xrf\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:09.868185 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.866748 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/935c23b1-953a-40cc-ad0c-bec2ef61afba-tls-certs\") pod \"distinct-w3-3048f9-kserve-5c85898b56-d6xrf\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:09.868185 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.866836 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-tmp-dir\") pod \"distinct-w3-3048f9-kserve-5c85898b56-d6xrf\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:09.868185 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.866871 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-model-cache\") pod \"distinct-w3-3048f9-kserve-5c85898b56-d6xrf\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:09.868185 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.866985 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-home\") pod \"distinct-w3-3048f9-kserve-5c85898b56-d6xrf\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:09.900571 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.900501 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:09:09.900917 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.900900 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:09:09.901089 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.901040 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5"] Oct 30 13:09:09.923165 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.923140 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:09:09.974877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.974844 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-tmp-dir\") pod \"distinct-w3-3048f9-kserve-5c85898b56-d6xrf\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:09.975126 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.975103 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-model-cache\") pod \"distinct-w3-3048f9-kserve-5c85898b56-d6xrf\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:09.975385 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.975370 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-home\") pod \"distinct-w3-3048f9-kserve-5c85898b56-d6xrf\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:09.975657 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.975644 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-dshm\") pod \"distinct-w3-3048f9-kserve-5c85898b56-d6xrf\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:09.975819 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.975804 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pfr8\" (UniqueName: \"kubernetes.io/projected/935c23b1-953a-40cc-ad0c-bec2ef61afba-kube-api-access-9pfr8\") pod \"distinct-w3-3048f9-kserve-5c85898b56-d6xrf\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:09.975924 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.975913 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/935c23b1-953a-40cc-ad0c-bec2ef61afba-tls-certs\") pod \"distinct-w3-3048f9-kserve-5c85898b56-d6xrf\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:09.977165 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.977049 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-tmp-dir\") pod \"distinct-w3-3048f9-kserve-5c85898b56-d6xrf\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:09.978508 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.977402 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-model-cache\") pod \"distinct-w3-3048f9-kserve-5c85898b56-d6xrf\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:09.978508 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:09.978219 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-home\") pod \"distinct-w3-3048f9-kserve-5c85898b56-d6xrf\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:10.011786 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:10.011693 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/935c23b1-953a-40cc-ad0c-bec2ef61afba-tls-certs\") pod \"distinct-w3-3048f9-kserve-5c85898b56-d6xrf\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:10.016790 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:10.014050 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:09:10.033753 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:10.032776 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pfr8\" (UniqueName: \"kubernetes.io/projected/935c23b1-953a-40cc-ad0c-bec2ef61afba-kube-api-access-9pfr8\") pod \"distinct-w3-3048f9-kserve-5c85898b56-d6xrf\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:10.061174 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:10.060918 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-dshm\") pod \"distinct-w3-3048f9-kserve-5c85898b56-d6xrf\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:10.294824 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:10.294742 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:10.560856 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:10.560770 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" event={"ID":"efa3fa51-d5f3-49c2-9658-378b1a6a2719","Type":"ContainerStarted","Data":"3d835359de3e69426b54ae7206758838d79f8a20a7fd38d7635f329a26904f71"} Oct 30 13:09:10.563051 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:10.563012 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" event={"ID":"617de128-a9b7-428c-acec-3753566fba40","Type":"ContainerStarted","Data":"b429d7932babad8e355fb75798ddedf556f8288ad0f8743f9cc6e3754b27001f"} Oct 30 13:09:10.563265 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:10.563103 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" event={"ID":"617de128-a9b7-428c-acec-3753566fba40","Type":"ContainerStarted","Data":"b39c196422216915701699e1bb2c12f258b2c5eb82958732e63b21ec2abf469c"} Oct 30 13:09:10.606821 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:10.606764 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" podStartSLOduration=5.606749027 podStartE2EDuration="5.606749027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:09:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:09:10.605166748 +0000 UTC m=+1049.630808012" watchObservedRunningTime="2026-10-30 13:09:10.606749027 +0000 UTC m=+1049.632390271" Oct 30 13:09:10.616468 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:09:10.616432 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod935c23b1_953a_40cc_ad0c_bec2ef61afba.slice/crio-6703f522eef94489618e043a1353aaac80d21aa41e151c138af63683837cdd54 WatchSource:0}: Error finding container 6703f522eef94489618e043a1353aaac80d21aa41e151c138af63683837cdd54: Status 404 returned error can't find the container with id 6703f522eef94489618e043a1353aaac80d21aa41e151c138af63683837cdd54 Oct 30 13:09:10.623812 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:10.623780 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf"] Oct 30 13:09:10.654033 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:10.653975 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" podStartSLOduration=5.653955558 podStartE2EDuration="5.653955558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:09:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:09:10.649809259 +0000 UTC m=+1049.675450534" watchObservedRunningTime="2026-10-30 13:09:10.653955558 +0000 UTC m=+1049.679596807" Oct 30 13:09:11.180088 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:11.178447 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:11.182444 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:11.182410 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 30 13:09:11.182660 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:11.182636 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" podUID="efa3fa51-d5f3-49c2-9658-378b1a6a2719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 30 13:09:11.288090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:11.284403 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:11.288090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:11.286850 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Oct 30 13:09:11.288090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:11.286906 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" podUID="617de128-a9b7-428c-acec-3753566fba40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Oct 30 13:09:11.668687 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:11.667988 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" event={"ID":"935c23b1-953a-40cc-ad0c-bec2ef61afba","Type":"ContainerStarted","Data":"1edced17b2f6ec44447f1f829686afd38fdd3b9f2dca386e345b7c03c5fd57ed"} Oct 30 13:09:11.668858 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:11.668751 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" event={"ID":"935c23b1-953a-40cc-ad0c-bec2ef61afba","Type":"ContainerStarted","Data":"6703f522eef94489618e043a1353aaac80d21aa41e151c138af63683837cdd54"} Oct 30 13:09:11.726513 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:11.726453 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" podStartSLOduration=2.726434975 podStartE2EDuration="2.726434975s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:09:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:09:11.725199677 +0000 UTC m=+1050.750840927" watchObservedRunningTime="2026-10-30 13:09:11.726434975 +0000 UTC m=+1050.752076224" Oct 30 13:09:12.185786 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:12.185752 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 30 13:09:12.186406 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:12.186373 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" podUID="efa3fa51-d5f3-49c2-9658-378b1a6a2719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 30 13:09:12.296090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:12.290162 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Oct 30 13:09:12.296090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:12.290217 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" podUID="617de128-a9b7-428c-acec-3753566fba40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Oct 30 13:09:12.304088 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:12.301157 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:13.185088 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:13.180415 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 30 13:09:13.185088 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:13.180475 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" podUID="efa3fa51-d5f3-49c2-9658-378b1a6a2719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 30 13:09:13.287503 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:13.287461 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Oct 30 13:09:13.288141 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:13.288107 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" podUID="617de128-a9b7-428c-acec-3753566fba40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Oct 30 13:09:14.182231 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:14.180423 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 30 13:09:14.182231 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:14.180479 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" podUID="efa3fa51-d5f3-49c2-9658-378b1a6a2719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 30 13:09:14.289090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:14.288421 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Oct 30 13:09:14.289090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:14.288478 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" podUID="617de128-a9b7-428c-acec-3753566fba40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Oct 30 13:09:15.030543 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:09:15.030505 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:09:15.178491 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:15.178430 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 30 13:09:15.178776 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:15.178742 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" podUID="efa3fa51-d5f3-49c2-9658-378b1a6a2719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 30 13:09:15.287537 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:15.287439 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Oct 30 13:09:15.287808 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:15.287778 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" podUID="617de128-a9b7-428c-acec-3753566fba40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Oct 30 13:09:16.179155 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:16.178427 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 30 13:09:16.179155 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:16.178500 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" podUID="efa3fa51-d5f3-49c2-9658-378b1a6a2719" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 30 13:09:16.290088 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:16.287480 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Oct 30 13:09:16.290088 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:16.287541 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" podUID="617de128-a9b7-428c-acec-3753566fba40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Oct 30 13:09:17.282779 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:17.282751 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:17.284573 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:17.284543 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:17.285784 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:17.285290 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Oct 30 13:09:17.285784 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:17.285338 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" podUID="617de128-a9b7-428c-acec-3753566fba40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Oct 30 13:09:17.286371 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:17.286059 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:09:18.288129 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:18.288092 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:18.288578 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:18.288559 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:18.289361 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:18.289338 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:09:20.295690 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:20.295658 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:20.309464 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:20.309438 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:20.796891 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:20.796859 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:09:31.789092 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:31.788200 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" podUID="99e695e7-e73c-429c-87ca-39ee740917c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:31.790092 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:31.789697 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" podUID="935c23b1-953a-40cc-ad0c-bec2ef61afba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:31.916086 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:31.915238 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" podUID="b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:31.916086 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:31.915625 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" podUID="19409b77-526f-48f7-b539-6d8ebb2d3dfc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded" Oct 30 13:09:31.940089 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:31.939157 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" podUID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:32.215225 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:32.215157 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" podUID="6962e138-faa9-438c-9138-1133bdce1011" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded" Oct 30 13:09:32.237428 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:32.237386 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" podUID="b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded" Oct 30 13:09:32.421240 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:32.421200 5029 patch_prober.go:29] interesting pod/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 13:09:32.421520 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:32.421489 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" podUID="6ef12288-2a06-4ad5-84ad-df2bcb9331d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:32.820220 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:32.820175 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" podUID="99e695e7-e73c-429c-87ca-39ee740917c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded" Oct 30 13:09:32.834091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:32.821717 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:09:32.843227 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:32.843149 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" podUID="2b878b27-465c-433d-aba4-e31849cfc14c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:32.981218 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:32.981171 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" podUID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:33.060107 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:32.985717 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" podUID="b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:33.065397 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:33.063802 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:09:33.065397 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:33.063875 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:09:33.065397 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:32.985750 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" podUID="19409b77-526f-48f7-b539-6d8ebb2d3dfc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded" Oct 30 13:09:33.065397 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:33.064724 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:09:33.189166 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:33.187258 5029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" podUID="2b878b27-465c-433d-aba4-e31849cfc14c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:33.235034 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:33.226160 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" podUID="6962e138-faa9-438c-9138-1133bdce1011" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded" Oct 30 13:09:33.260153 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:33.238139 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" podUID="b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded" Oct 30 13:09:33.507144 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:33.505217 5029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" podUID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:34.067098 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:34.067035 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" podUID="99e695e7-e73c-429c-87ca-39ee740917c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded" Oct 30 13:09:34.179092 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:34.155301 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" podUID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded" Oct 30 13:09:34.179740 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:34.161218 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" podUID="b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/ready\": context deadline exceeded" Oct 30 13:09:34.214090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:34.161341 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" podUID="19409b77-526f-48f7-b539-6d8ebb2d3dfc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:34.367158 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:34.351081 5029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" podUID="19409b77-526f-48f7-b539-6d8ebb2d3dfc" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/health\": context deadline exceeded" Oct 30 13:09:34.387183 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:34.387138 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" podUID="6962e138-faa9-438c-9138-1133bdce1011" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:34.394523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:34.394487 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" podUID="b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded" Oct 30 13:09:34.446091 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:09:34.401830 5029 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.32s" Oct 30 13:09:34.462139 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:34.460545 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:09:34.462408 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:34.462391 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:09:34.501757 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:34.501726 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:09:34.502924 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:34.502904 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:09:34.679185 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:34.679084 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" podUID="d6bdcd3f-0200-48e6-af85-2d1d8f022054" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:34.876910 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:34.876864 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:09:34.896729 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:34.887332 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" podUID="935c23b1-953a-40cc-ad0c-bec2ef61afba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:09:34.951051 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:34.950973 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:09:35.038923 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:35.038894 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:09:35.040113 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:35.039364 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:09:38.178359 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.178325 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t"] Oct 30 13:09:38.183098 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.183044 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:38.202746 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.202719 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t"] Oct 30 13:09:38.210464 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.210439 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-42ee9d\"/\"distinct-2-w0-42ee9d-kserve-self-signed-certs\"" Oct 30 13:09:38.279270 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.279235 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-tmp-dir\") pod \"distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:38.279445 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.279282 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-home\") pod \"distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:38.279445 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.279313 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-dshm\") pod \"distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:38.279528 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.279457 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-model-cache\") pod \"distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:38.279528 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.279490 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnshr\" (UniqueName: \"kubernetes.io/projected/607535ec-5cbf-4b96-a053-6519ddc2302b-kube-api-access-pnshr\") pod \"distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:38.279528 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.279525 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/607535ec-5cbf-4b96-a053-6519ddc2302b-tls-certs\") pod \"distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:38.382991 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.382949 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-model-cache\") pod \"distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:38.383838 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.383811 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnshr\" (UniqueName: \"kubernetes.io/projected/607535ec-5cbf-4b96-a053-6519ddc2302b-kube-api-access-pnshr\") pod \"distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:38.384028 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.384010 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/607535ec-5cbf-4b96-a053-6519ddc2302b-tls-certs\") pod \"distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:38.384292 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.384271 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-tmp-dir\") pod \"distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:38.384443 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.384428 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-home\") pod \"distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:38.384580 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.384568 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-dshm\") pod \"distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:38.385310 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.384977 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-tmp-dir\") pod \"distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:38.385310 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.383732 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-model-cache\") pod \"distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:38.385310 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.385276 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-home\") pod \"distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:38.391423 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.391402 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-dshm\") pod \"distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:38.392602 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.392579 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/607535ec-5cbf-4b96-a053-6519ddc2302b-tls-certs\") pod \"distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:38.410665 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.410632 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnshr\" (UniqueName: \"kubernetes.io/projected/607535ec-5cbf-4b96-a053-6519ddc2302b-kube-api-access-pnshr\") pod \"distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:38.498096 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.497973 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:38.827292 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:38.827265 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t"] Oct 30 13:09:38.828252 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:09:38.828225 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod607535ec_5cbf_4b96_a053_6519ddc2302b.slice/crio-abe6c182468bed9ad11f8f69e43ad0e7d6fa8e877c75e12bce1025e388d1c147 WatchSource:0}: Error finding container abe6c182468bed9ad11f8f69e43ad0e7d6fa8e877c75e12bce1025e388d1c147: Status 404 returned error can't find the container with id abe6c182468bed9ad11f8f69e43ad0e7d6fa8e877c75e12bce1025e388d1c147 Oct 30 13:09:39.142390 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:39.142298 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" event={"ID":"607535ec-5cbf-4b96-a053-6519ddc2302b","Type":"ContainerStarted","Data":"4c465c0a709deeb88d129f0396969c77447803a4bd24c3fbe714b8a04afa272a"} Oct 30 13:09:39.142390 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:39.142347 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" event={"ID":"607535ec-5cbf-4b96-a053-6519ddc2302b","Type":"ContainerStarted","Data":"abe6c182468bed9ad11f8f69e43ad0e7d6fa8e877c75e12bce1025e388d1c147"} Oct 30 13:09:39.188126 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:39.188049 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" podStartSLOduration=1.18803614 podStartE2EDuration="1.18803614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:09:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:09:39.184506642 +0000 UTC m=+1078.210147921" watchObservedRunningTime="2026-10-30 13:09:39.18803614 +0000 UTC m=+1078.213677406" Oct 30 13:09:39.498467 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:39.498387 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:44.880196 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:09:44.880159 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:09:46.154978 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:46.154942 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh"] Oct 30 13:09:46.160551 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:46.160528 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh" Oct 30 13:09:46.185093 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:46.185043 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh"] Oct 30 13:09:46.203912 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:46.203859 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-253f7c37-dockercfg-hh2hz\"" Oct 30 13:09:46.211735 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:46.211690 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-253f7c37\"" Oct 30 13:09:46.319159 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:46.319126 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6-plugins-config-volume\") pod \"payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh\" (UID: \"5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6\") " pod="openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh" Oct 30 13:09:46.319438 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:46.319408 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bmbp\" (UniqueName: \"kubernetes.io/projected/5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6-kube-api-access-7bmbp\") pod \"payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh\" (UID: \"5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6\") " pod="openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh" Oct 30 13:09:46.421168 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:46.421090 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bmbp\" (UniqueName: \"kubernetes.io/projected/5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6-kube-api-access-7bmbp\") pod \"payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh\" (UID: \"5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6\") " pod="openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh" Oct 30 13:09:46.421318 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:46.421191 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6-plugins-config-volume\") pod \"payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh\" (UID: \"5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6\") " pod="openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh" Oct 30 13:09:46.421950 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:46.421913 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6-plugins-config-volume\") pod \"payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh\" (UID: \"5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6\") " pod="openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh" Oct 30 13:09:46.445529 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:46.445500 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bmbp\" (UniqueName: \"kubernetes.io/projected/5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6-kube-api-access-7bmbp\") pod \"payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh\" (UID: \"5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6\") " pod="openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh" Oct 30 13:09:46.483622 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:46.483583 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh" Oct 30 13:09:46.791221 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:46.791197 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh"] Oct 30 13:09:46.794487 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:09:46.794462 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d7c6015_cbd6_4b2c_ace9_dd9a73a001b6.slice/crio-4b2b888309678162ac9b2f6cec7122ffa9524b4f9b6ad261f8e564290a3f0b70 WatchSource:0}: Error finding container 4b2b888309678162ac9b2f6cec7122ffa9524b4f9b6ad261f8e564290a3f0b70: Status 404 returned error can't find the container with id 4b2b888309678162ac9b2f6cec7122ffa9524b4f9b6ad261f8e564290a3f0b70 Oct 30 13:09:46.797027 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:46.797010 5029 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 13:09:47.218127 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:47.218060 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh" event={"ID":"5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6","Type":"ContainerStarted","Data":"4b2b888309678162ac9b2f6cec7122ffa9524b4f9b6ad261f8e564290a3f0b70"} Oct 30 13:09:48.498470 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:48.498413 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:48.513482 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:48.513451 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:49.235617 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:49.235577 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh" event={"ID":"5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6","Type":"ContainerStarted","Data":"cc89e1cf5f89ec71579e0e1ff3f3c8958085af8212a45fecb90fc80aa3a3d665"} Oct 30 13:09:49.235964 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:49.235939 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh" Oct 30 13:09:49.246889 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:49.246870 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:09:49.302002 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:49.301939 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh" podStartSLOduration=1.288013946 podStartE2EDuration="3.301906807s" totalImagesPullingTime="2.013892861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:09:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:09:49.289686034 +0000 UTC m=+1088.315327302" watchObservedRunningTime="2026-10-30 13:09:49.301906807 +0000 UTC m=+1088.327548058" Oct 30 13:09:50.697295 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.697253 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6"] Oct 30 13:09:50.702246 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.702221 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.730142 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.730104 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx"] Oct 30 13:09:50.735342 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.735319 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.738715 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.738688 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6"] Oct 30 13:09:50.766638 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.766605 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx"] Oct 30 13:09:50.823091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.822767 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-istio-envoy\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.823091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.822835 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-istio-data\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.823091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.822875 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istiod-ca-cert\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.823091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.822924 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-envoy\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.823091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.822975 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-podinfo\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.823091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.823008 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-workload-certs\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.823091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.823053 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6328797e-74fc-4877-acac-0554466d541a-istiod-ca-cert\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.823736 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.823137 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-credential-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.823736 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.823166 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2blz7\" (UniqueName: \"kubernetes.io/projected/6328797e-74fc-4877-acac-0554466d541a-kube-api-access-2blz7\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.823736 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.823202 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-data\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.823736 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.823231 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvh46\" (UniqueName: \"kubernetes.io/projected/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-kube-api-access-kvh46\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.823736 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.823327 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-credential-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.823736 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.823377 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6328797e-74fc-4877-acac-0554466d541a-istio-token\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.823736 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.823421 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-workload-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.823736 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.823446 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6328797e-74fc-4877-acac-0554466d541a-istio-podinfo\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.823736 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.823472 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-workload-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.823736 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.823496 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-token\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.823736 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.823543 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-workload-certs\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.925438 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.925318 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-istio-data\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.925438 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.925375 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istiod-ca-cert\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.925438 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.925424 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-envoy\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.925438 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.925463 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-podinfo\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.925828 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.925488 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-workload-certs\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.925828 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.925524 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6328797e-74fc-4877-acac-0554466d541a-istiod-ca-cert\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.925828 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.925575 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-credential-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.925828 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.925592 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2blz7\" (UniqueName: \"kubernetes.io/projected/6328797e-74fc-4877-acac-0554466d541a-kube-api-access-2blz7\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.925828 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.925625 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-data\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.925828 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.925691 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvh46\" (UniqueName: \"kubernetes.io/projected/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-kube-api-access-kvh46\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.925828 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.925808 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-credential-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.926244 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.925861 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6328797e-74fc-4877-acac-0554466d541a-istio-token\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.926244 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.925910 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-istio-data\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.926244 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.925913 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-workload-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.926244 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.925985 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6328797e-74fc-4877-acac-0554466d541a-istio-podinfo\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.926244 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.926016 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-workload-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.926244 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.926042 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-token\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.926244 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.926105 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-workload-certs\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.926244 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.926151 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-istio-envoy\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.926650 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.926294 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-workload-certs\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.926650 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.926456 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-workload-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.926650 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.926418 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-data\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.926650 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.926533 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istiod-ca-cert\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.926868 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.926678 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-credential-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.926868 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.926809 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-workload-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.927013 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.926988 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-credential-socket\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.927408 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.927191 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-workload-certs\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.927408 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.927231 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6328797e-74fc-4877-acac-0554466d541a-istiod-ca-cert\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.929117 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.929094 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-istio-envoy\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.929526 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.929499 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-envoy\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.929786 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.929772 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-podinfo\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.930191 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.930169 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6328797e-74fc-4877-acac-0554466d541a-istio-podinfo\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.950370 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.950304 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-token\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:50.950499 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.950480 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2blz7\" (UniqueName: \"kubernetes.io/projected/6328797e-74fc-4877-acac-0554466d541a-kube-api-access-2blz7\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.950563 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.950496 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6328797e-74fc-4877-acac-0554466d541a-istio-token\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:50.951111 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:50.951056 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvh46\" (UniqueName: \"kubernetes.io/projected/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-kube-api-access-kvh46\") pod \"e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:51.019930 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:51.019898 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:51.064144 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:51.064109 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:51.346959 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:51.346927 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6"] Oct 30 13:09:51.350401 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:09:51.350368 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6328797e_74fc_4877_acac_0554466d541a.slice/crio-cbeed0e47cb3cddf9a65b3b7bbf321c8d8f77a1b4391e3ed4699924eddff8205 WatchSource:0}: Error finding container cbeed0e47cb3cddf9a65b3b7bbf321c8d8f77a1b4391e3ed4699924eddff8205: Status 404 returned error can't find the container with id cbeed0e47cb3cddf9a65b3b7bbf321c8d8f77a1b4391e3ed4699924eddff8205 Oct 30 13:09:51.353000 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:51.352965 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:09:51.353131 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:51.353034 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:09:51.353131 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:51.353083 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:09:51.397091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:51.396957 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx"] Oct 30 13:09:51.399486 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:09:51.399454 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd2c4ae1_fbe5_46c9_82c7_f4f418994e31.slice/crio-8a68fa25957d03055387234a9d091045500e33afa74006a3d0549776dd2b573f WatchSource:0}: Error finding container 8a68fa25957d03055387234a9d091045500e33afa74006a3d0549776dd2b573f: Status 404 returned error can't find the container with id 8a68fa25957d03055387234a9d091045500e33afa74006a3d0549776dd2b573f Oct 30 13:09:51.402353 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:51.402323 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:09:51.402432 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:51.402389 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:09:51.402469 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:51.402431 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:09:52.258139 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:52.258104 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" event={"ID":"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31","Type":"ContainerStarted","Data":"ddd3d6dd30cbd234af3fe4d9dfdf59a21e4235a0371106bb139d83d03ea6e78e"} Oct 30 13:09:52.258552 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:52.258161 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" event={"ID":"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31","Type":"ContainerStarted","Data":"8a68fa25957d03055387234a9d091045500e33afa74006a3d0549776dd2b573f"} Oct 30 13:09:52.259823 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:52.259798 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" event={"ID":"6328797e-74fc-4877-acac-0554466d541a","Type":"ContainerStarted","Data":"a963497172e5ae9baaaf5151b01a4559180508786066a5c10b6e197013912da6"} Oct 30 13:09:52.259929 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:52.259834 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" event={"ID":"6328797e-74fc-4877-acac-0554466d541a","Type":"ContainerStarted","Data":"cbeed0e47cb3cddf9a65b3b7bbf321c8d8f77a1b4391e3ed4699924eddff8205"} Oct 30 13:09:52.315983 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:52.315924 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" podStartSLOduration=2.315909695 podStartE2EDuration="2.315909695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:09:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:09:52.315659972 +0000 UTC m=+1091.341301240" watchObservedRunningTime="2026-10-30 13:09:52.315909695 +0000 UTC m=+1091.341550942" Oct 30 13:09:53.021436 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:53.021402 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:53.022961 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:53.022928 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 13:09:53.023200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:53.023172 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" podUID="6328797e-74fc-4877-acac-0554466d541a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 13:09:53.064522 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:53.064496 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:53.065727 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:53.065709 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 30 13:09:53.065790 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:53.065749 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" podUID="fd2c4ae1-fbe5-46c9-82c7-f4f418994e31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 30 13:09:54.024508 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:54.024469 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 13:09:54.025112 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:54.025057 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" podUID="6328797e-74fc-4877-acac-0554466d541a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 13:09:54.065150 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:54.065118 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 30 13:09:54.065309 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:54.065176 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" podUID="fd2c4ae1-fbe5-46c9-82c7-f4f418994e31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 30 13:09:55.021088 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:55.020600 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 13:09:55.021088 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:55.020680 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" podUID="6328797e-74fc-4877-acac-0554466d541a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 13:09:55.065236 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:55.065193 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 30 13:09:55.065624 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:55.065258 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" podUID="fd2c4ae1-fbe5-46c9-82c7-f4f418994e31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 30 13:09:56.022885 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:56.022849 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 13:09:56.023203 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:56.023171 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" podUID="6328797e-74fc-4877-acac-0554466d541a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 13:09:56.064714 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:56.064689 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 30 13:09:56.064913 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:56.064881 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" podUID="fd2c4ae1-fbe5-46c9-82c7-f4f418994e31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 30 13:09:57.030447 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:57.030122 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 13:09:57.030447 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:57.030179 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" podUID="6328797e-74fc-4877-acac-0554466d541a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 13:09:57.065975 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:57.064632 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 30 13:09:57.065975 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:57.064718 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" podUID="fd2c4ae1-fbe5-46c9-82c7-f4f418994e31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 30 13:09:58.020465 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:58.020427 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 13:09:58.020632 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:58.020489 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" podUID="6328797e-74fc-4877-acac-0554466d541a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 13:09:58.065402 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:58.065366 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 30 13:09:58.065747 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:58.065429 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" podUID="fd2c4ae1-fbe5-46c9-82c7-f4f418994e31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 30 13:09:59.024399 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:59.024370 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:59.024798 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:59.024772 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:59.025588 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:59.025567 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:09:59.069903 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:59.069873 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:59.070380 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:59.070167 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:59.070882 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:59.070864 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:09:59.071079 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:09:59.071026 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" podStartSLOduration=9.07101408 podStartE2EDuration="9.07101408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:09:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:09:52.385378343 +0000 UTC m=+1091.411019617" watchObservedRunningTime="2026-10-30 13:09:59.07101408 +0000 UTC m=+1098.096655327" Oct 30 13:10:00.247834 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:00.247808 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh" Oct 30 13:10:00.793217 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:00.793182 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r"] Oct 30 13:10:00.798301 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:00.798275 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:00.826682 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:00.826652 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r"] Oct 30 13:10:00.826795 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:00.826722 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-3048f9\"/\"distinct-2-w3-3048f9-kserve-self-signed-certs\"" Oct 30 13:10:00.956710 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:00.956665 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9af7cbdb-5aee-4f11-9446-fceacd3fb135-tls-certs\") pod \"distinct-2-w3-3048f9-kserve-5954dc55-x7p7r\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:00.956913 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:00.956786 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-dshm\") pod \"distinct-2-w3-3048f9-kserve-5954dc55-x7p7r\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:00.956913 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:00.956819 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-tmp-dir\") pod \"distinct-2-w3-3048f9-kserve-5954dc55-x7p7r\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:00.956913 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:00.956873 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qzfp\" (UniqueName: \"kubernetes.io/projected/9af7cbdb-5aee-4f11-9446-fceacd3fb135-kube-api-access-6qzfp\") pod \"distinct-2-w3-3048f9-kserve-5954dc55-x7p7r\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:00.957107 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:00.957024 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-home\") pod \"distinct-2-w3-3048f9-kserve-5954dc55-x7p7r\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:00.957210 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:00.957189 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-model-cache\") pod \"distinct-2-w3-3048f9-kserve-5954dc55-x7p7r\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:01.059864 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:01.059770 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-model-cache\") pod \"distinct-2-w3-3048f9-kserve-5954dc55-x7p7r\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:01.060116 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:01.059938 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9af7cbdb-5aee-4f11-9446-fceacd3fb135-tls-certs\") pod \"distinct-2-w3-3048f9-kserve-5954dc55-x7p7r\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:01.060116 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:01.060001 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-dshm\") pod \"distinct-2-w3-3048f9-kserve-5954dc55-x7p7r\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:01.060116 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:01.060024 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-tmp-dir\") pod \"distinct-2-w3-3048f9-kserve-5954dc55-x7p7r\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:01.060116 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:01.060107 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qzfp\" (UniqueName: \"kubernetes.io/projected/9af7cbdb-5aee-4f11-9446-fceacd3fb135-kube-api-access-6qzfp\") pod \"distinct-2-w3-3048f9-kserve-5954dc55-x7p7r\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:01.060362 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:01.060174 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-home\") pod \"distinct-2-w3-3048f9-kserve-5954dc55-x7p7r\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:01.060362 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:01.060355 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-model-cache\") pod \"distinct-2-w3-3048f9-kserve-5954dc55-x7p7r\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:01.060560 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:01.060540 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-home\") pod \"distinct-2-w3-3048f9-kserve-5954dc55-x7p7r\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:01.060804 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:01.060782 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-tmp-dir\") pod \"distinct-2-w3-3048f9-kserve-5954dc55-x7p7r\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:01.079118 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:01.078431 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-dshm\") pod \"distinct-2-w3-3048f9-kserve-5954dc55-x7p7r\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:01.080908 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:01.080888 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9af7cbdb-5aee-4f11-9446-fceacd3fb135-tls-certs\") pod \"distinct-2-w3-3048f9-kserve-5954dc55-x7p7r\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:01.088786 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:01.088748 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qzfp\" (UniqueName: \"kubernetes.io/projected/9af7cbdb-5aee-4f11-9446-fceacd3fb135-kube-api-access-6qzfp\") pod \"distinct-2-w3-3048f9-kserve-5954dc55-x7p7r\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:01.116697 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:01.116664 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:01.450384 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:01.450354 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r"] Oct 30 13:10:01.453245 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:10:01.453218 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9af7cbdb_5aee_4f11_9446_fceacd3fb135.slice/crio-7f6c05b181f68cdc85cd65e6440505182271e5fc3bd5d642a6b3ded7739191b8 WatchSource:0}: Error finding container 7f6c05b181f68cdc85cd65e6440505182271e5fc3bd5d642a6b3ded7739191b8: Status 404 returned error can't find the container with id 7f6c05b181f68cdc85cd65e6440505182271e5fc3bd5d642a6b3ded7739191b8 Oct 30 13:10:02.393857 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:02.393812 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" event={"ID":"9af7cbdb-5aee-4f11-9446-fceacd3fb135","Type":"ContainerStarted","Data":"f7b79dab050cc1d07509dee3eafa8236eaef2eb354f2bd1e37f55e68208257e0"} Oct 30 13:10:02.393857 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:02.393864 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" event={"ID":"9af7cbdb-5aee-4f11-9446-fceacd3fb135","Type":"ContainerStarted","Data":"7f6c05b181f68cdc85cd65e6440505182271e5fc3bd5d642a6b3ded7739191b8"} Oct 30 13:10:02.439419 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:02.439372 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" podStartSLOduration=2.439357495 podStartE2EDuration="2.439357495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:10:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:10:02.434180222 +0000 UTC m=+1101.459821491" watchObservedRunningTime="2026-10-30 13:10:02.439357495 +0000 UTC m=+1101.464998742" Oct 30 13:10:03.116980 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:03.116911 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:05.544836 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.544805 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp"] Oct 30 13:10:05.565263 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.565233 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.583306 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.583276 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp"] Oct 30 13:10:05.629782 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.629739 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-envoy\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.629974 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.629792 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftwxk\" (UniqueName: \"kubernetes.io/projected/ac96e53b-9dec-4b7a-bd58-833239583b83-kube-api-access-ftwxk\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.629974 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.629904 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-data\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.629974 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.629942 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-workload-socket\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.630135 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.629975 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-token\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.630135 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.630094 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-credential-socket\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.630214 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.630199 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac96e53b-9dec-4b7a-bd58-833239583b83-istiod-ca-cert\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.630271 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.630258 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-podinfo\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.630348 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.630327 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-workload-certs\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.731731 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.731691 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-credential-socket\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.731920 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.731802 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac96e53b-9dec-4b7a-bd58-833239583b83-istiod-ca-cert\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.731920 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.731866 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-podinfo\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.731920 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.731913 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-workload-certs\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.732110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.731977 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-envoy\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.732110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.732018 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftwxk\" (UniqueName: \"kubernetes.io/projected/ac96e53b-9dec-4b7a-bd58-833239583b83-kube-api-access-ftwxk\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.732231 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.732115 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-data\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.732231 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.732143 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-workload-socket\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.732231 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.732165 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-credential-socket\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.732231 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.732181 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-token\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.732430 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.732273 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-workload-certs\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.732488 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.732452 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-workload-socket\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.732535 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.732503 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-data\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.732668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.732645 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac96e53b-9dec-4b7a-bd58-833239583b83-istiod-ca-cert\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.734470 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.734439 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-podinfo\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.734572 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.734546 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-envoy\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.756472 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.756442 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-token\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.756602 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.756583 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftwxk\" (UniqueName: \"kubernetes.io/projected/ac96e53b-9dec-4b7a-bd58-833239583b83-kube-api-access-ftwxk\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:05.886747 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:05.886712 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:06.210655 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:06.210631 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp"] Oct 30 13:10:06.213532 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:10:06.213506 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac96e53b_9dec_4b7a_bd58_833239583b83.slice/crio-08bea501eb353f76966101ddb0dcf778f6015ed97fc949f8c2d7dc47c26321e2 WatchSource:0}: Error finding container 08bea501eb353f76966101ddb0dcf778f6015ed97fc949f8c2d7dc47c26321e2: Status 404 returned error can't find the container with id 08bea501eb353f76966101ddb0dcf778f6015ed97fc949f8c2d7dc47c26321e2 Oct 30 13:10:06.216109 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:06.216041 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:10:06.216188 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:06.216143 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:10:06.216188 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:06.216174 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:10:06.420397 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:06.420361 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" event={"ID":"ac96e53b-9dec-4b7a-bd58-833239583b83","Type":"ContainerStarted","Data":"8f1b3e7f027ed732bc0c4f9fd317e8648356160e5f507533e9ebcceb3089197f"} Oct 30 13:10:06.420584 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:06.420427 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" event={"ID":"ac96e53b-9dec-4b7a-bd58-833239583b83","Type":"ContainerStarted","Data":"08bea501eb353f76966101ddb0dcf778f6015ed97fc949f8c2d7dc47c26321e2"} Oct 30 13:10:06.466844 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:06.466746 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" podStartSLOduration=1.466731809 podStartE2EDuration="1.466731809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:10:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:10:06.464167906 +0000 UTC m=+1105.489809175" watchObservedRunningTime="2026-10-30 13:10:06.466731809 +0000 UTC m=+1105.492373057" Oct 30 13:10:06.886869 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:06.886828 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:07.897517 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:07.897481 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body= Oct 30 13:10:07.897910 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:07.897548 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" podUID="ac96e53b-9dec-4b7a-bd58-833239583b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" Oct 30 13:10:08.892290 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:08.892165 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body= Oct 30 13:10:08.892290 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:08.892227 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" podUID="ac96e53b-9dec-4b7a-bd58-833239583b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" Oct 30 13:10:09.896148 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:09.896108 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body= Oct 30 13:10:09.896721 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:09.896176 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" podUID="ac96e53b-9dec-4b7a-bd58-833239583b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" Oct 30 13:10:10.896091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:10.891158 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body= Oct 30 13:10:10.896091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:10.891230 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" podUID="ac96e53b-9dec-4b7a-bd58-833239583b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" Oct 30 13:10:11.117724 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:11.117692 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:11.131273 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:11.131247 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:11.465941 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:11.465905 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:10:11.891590 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:11.891494 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body= Oct 30 13:10:11.891913 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:11.891872 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" podUID="ac96e53b-9dec-4b7a-bd58-833239583b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" Oct 30 13:10:12.887405 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:12.887373 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body= Oct 30 13:10:12.887934 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:12.887429 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" podUID="ac96e53b-9dec-4b7a-bd58-833239583b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" Oct 30 13:10:13.891097 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:13.890479 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body= Oct 30 13:10:13.891097 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:13.890555 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" podUID="ac96e53b-9dec-4b7a-bd58-833239583b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" Oct 30 13:10:14.470199 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:14.470152 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" podUID="9af7cbdb-5aee-4f11-9446-fceacd3fb135" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:10:14.640209 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:14.640159 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" podUID="d6bdcd3f-0200-48e6-af85-2d1d8f022054" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded" Oct 30 13:10:14.899207 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:14.899160 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" podUID="6962e138-faa9-438c-9138-1133bdce1011" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:10:15.075299 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:10:15.075258 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:10:15.087290 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:15.087185 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 13:10:15.087624 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:15.087589 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" podUID="6328797e-74fc-4877-acac-0554466d541a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 13:10:15.140204 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:15.140166 5029 patch_prober.go:29] interesting pod/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 13:10:15.140464 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:15.140439 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" podUID="fd2c4ae1-fbe5-46c9-82c7-f4f418994e31" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 13:10:15.184698 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:15.184575 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" podUID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:10:15.241444 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:15.241396 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" podUID="607535ec-5cbf-4b96-a053-6519ddc2302b" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:10:15.673199 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:15.673152 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" podUID="d6bdcd3f-0200-48e6-af85-2d1d8f022054" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:10:15.715089 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:15.674163 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:10:15.881662 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:15.881633 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:15.887167 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:15.817492 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" podUID="935c23b1-953a-40cc-ad0c-bec2ef61afba" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded" Oct 30 13:10:15.887471 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:15.816465 5029 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" podUID="b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.63:8000/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 13:10:15.887889 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:15.887867 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:15.905521 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:15.905492 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:10:15.926133 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:15.925683 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:10:20.616362 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.616330 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27"] Oct 30 13:10:20.620750 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.620731 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.645188 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.645158 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27"] Oct 30 13:10:20.742417 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.742384 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-workload-socket\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.742603 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.742430 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-istio-envoy\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.742678 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.742604 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-credential-socket\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.742733 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.742675 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b80d9479-2325-40f0-93fd-329c462b629c-istio-token\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.742843 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.742821 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b80d9479-2325-40f0-93fd-329c462b629c-istio-podinfo\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.742905 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.742873 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-workload-certs\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.743008 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.742992 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnnn4\" (UniqueName: \"kubernetes.io/projected/b80d9479-2325-40f0-93fd-329c462b629c-kube-api-access-lnnn4\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.743146 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.743129 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b80d9479-2325-40f0-93fd-329c462b629c-istiod-ca-cert\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.743217 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.743164 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-istio-data\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.844899 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.844862 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b80d9479-2325-40f0-93fd-329c462b629c-istiod-ca-cert\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.844899 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.844903 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-istio-data\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.845153 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.845027 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-workload-socket\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.845153 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.845118 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-istio-envoy\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.845279 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.845263 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-credential-socket\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.845337 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.845297 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-istio-data\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.845337 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.845329 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b80d9479-2325-40f0-93fd-329c462b629c-istio-token\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.845439 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.845396 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b80d9479-2325-40f0-93fd-329c462b629c-istio-podinfo\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.845439 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.845422 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-workload-socket\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.845539 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.845442 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-workload-certs\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.845539 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.845524 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnnn4\" (UniqueName: \"kubernetes.io/projected/b80d9479-2325-40f0-93fd-329c462b629c-kube-api-access-lnnn4\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.845646 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.845574 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-credential-socket\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.845711 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.845672 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b80d9479-2325-40f0-93fd-329c462b629c-istiod-ca-cert\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.845902 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.845876 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-workload-certs\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.847686 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.847662 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b80d9479-2325-40f0-93fd-329c462b629c-istio-podinfo\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.847774 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.847662 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-istio-envoy\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.869288 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.869234 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b80d9479-2325-40f0-93fd-329c462b629c-istio-token\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.869593 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.869572 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnnn4\" (UniqueName: \"kubernetes.io/projected/b80d9479-2325-40f0-93fd-329c462b629c-kube-api-access-lnnn4\") pod \"e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:20.936217 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:20.936189 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:21.269674 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:21.269646 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27"] Oct 30 13:10:21.272035 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:10:21.272002 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb80d9479_2325_40f0_93fd_329c462b629c.slice/crio-47829c404d0bbe9a604bcbff36265cabb5caa2329cc6ba130ebf5cdccd0e2c18 WatchSource:0}: Error finding container 47829c404d0bbe9a604bcbff36265cabb5caa2329cc6ba130ebf5cdccd0e2c18: Status 404 returned error can't find the container with id 47829c404d0bbe9a604bcbff36265cabb5caa2329cc6ba130ebf5cdccd0e2c18 Oct 30 13:10:21.274605 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:21.274570 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:10:21.274687 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:21.274636 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:10:21.274687 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:21.274666 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:10:21.932601 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:21.932566 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" event={"ID":"b80d9479-2325-40f0-93fd-329c462b629c","Type":"ContainerStarted","Data":"78bde478d6fe6552fa74543291ed1528f79c109e3d1c3612476feec179826218"} Oct 30 13:10:21.933010 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:21.932624 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" event={"ID":"b80d9479-2325-40f0-93fd-329c462b629c","Type":"ContainerStarted","Data":"47829c404d0bbe9a604bcbff36265cabb5caa2329cc6ba130ebf5cdccd0e2c18"} Oct 30 13:10:21.936899 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:21.936877 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:21.980332 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:21.980284 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" podStartSLOduration=1.98027083 podStartE2EDuration="1.98027083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:10:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:10:21.977196318 +0000 UTC m=+1121.002837588" watchObservedRunningTime="2026-10-30 13:10:21.98027083 +0000 UTC m=+1121.005912077" Oct 30 13:10:22.938653 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:22.938624 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 30 13:10:22.939030 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:22.938674 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" podUID="b80d9479-2325-40f0-93fd-329c462b629c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 30 13:10:23.936931 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:23.936895 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 30 13:10:23.937127 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:23.936950 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" podUID="b80d9479-2325-40f0-93fd-329c462b629c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 30 13:10:24.936999 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:24.936963 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 30 13:10:24.937431 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:24.937023 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" podUID="b80d9479-2325-40f0-93fd-329c462b629c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 30 13:10:25.937200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:25.937167 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 30 13:10:25.937564 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:25.937217 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" podUID="b80d9479-2325-40f0-93fd-329c462b629c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 30 13:10:26.937208 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:26.937168 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 30 13:10:26.937710 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:26.937420 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" podUID="b80d9479-2325-40f0-93fd-329c462b629c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 30 13:10:27.937701 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:27.937662 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 30 13:10:27.938253 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:27.937728 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" podUID="b80d9479-2325-40f0-93fd-329c462b629c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 30 13:10:29.299222 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:29.299185 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:29.302427 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:29.302400 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:29.307370 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:29.307339 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:10:40.529017 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.528980 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr"] Oct 30 13:10:40.534351 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.534331 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:40.549202 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.549174 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr"] Oct 30 13:10:40.555527 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.555500 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-ac0c58\"/\"distinct-w1-ac0c58-kserve-self-signed-certs\"" Oct 30 13:10:40.619564 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.619528 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-model-cache\") pod \"distinct-w1-ac0c58-kserve-dccc84448-zjdfr\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:40.619891 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.619621 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-tmp-dir\") pod \"distinct-w1-ac0c58-kserve-dccc84448-zjdfr\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:40.619891 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.619765 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q4cr\" (UniqueName: \"kubernetes.io/projected/06d7d761-d47f-4aa5-b931-46f910b65597-kube-api-access-4q4cr\") pod \"distinct-w1-ac0c58-kserve-dccc84448-zjdfr\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:40.619891 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.619869 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-dshm\") pod \"distinct-w1-ac0c58-kserve-dccc84448-zjdfr\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:40.620087 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.619915 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-home\") pod \"distinct-w1-ac0c58-kserve-dccc84448-zjdfr\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:40.620087 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.619939 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06d7d761-d47f-4aa5-b931-46f910b65597-tls-certs\") pod \"distinct-w1-ac0c58-kserve-dccc84448-zjdfr\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:40.721620 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.721581 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q4cr\" (UniqueName: \"kubernetes.io/projected/06d7d761-d47f-4aa5-b931-46f910b65597-kube-api-access-4q4cr\") pod \"distinct-w1-ac0c58-kserve-dccc84448-zjdfr\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:40.722427 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.722400 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-dshm\") pod \"distinct-w1-ac0c58-kserve-dccc84448-zjdfr\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:40.723468 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.723446 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-home\") pod \"distinct-w1-ac0c58-kserve-dccc84448-zjdfr\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:40.723584 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.723493 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-home\") pod \"distinct-w1-ac0c58-kserve-dccc84448-zjdfr\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:40.723584 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.723528 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06d7d761-d47f-4aa5-b931-46f910b65597-tls-certs\") pod \"distinct-w1-ac0c58-kserve-dccc84448-zjdfr\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:40.723724 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.723651 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-model-cache\") pod \"distinct-w1-ac0c58-kserve-dccc84448-zjdfr\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:40.723780 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.723753 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-tmp-dir\") pod \"distinct-w1-ac0c58-kserve-dccc84448-zjdfr\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:40.724135 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.724115 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-tmp-dir\") pod \"distinct-w1-ac0c58-kserve-dccc84448-zjdfr\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:40.726476 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.725009 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-model-cache\") pod \"distinct-w1-ac0c58-kserve-dccc84448-zjdfr\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:40.729709 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.729685 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06d7d761-d47f-4aa5-b931-46f910b65597-tls-certs\") pod \"distinct-w1-ac0c58-kserve-dccc84448-zjdfr\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:40.735100 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.733898 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-dshm\") pod \"distinct-w1-ac0c58-kserve-dccc84448-zjdfr\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:40.751954 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.751918 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q4cr\" (UniqueName: \"kubernetes.io/projected/06d7d761-d47f-4aa5-b931-46f910b65597-kube-api-access-4q4cr\") pod \"distinct-w1-ac0c58-kserve-dccc84448-zjdfr\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:40.851194 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:40.851153 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:41.197768 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:41.197672 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr"] Oct 30 13:10:41.202272 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:10:41.202245 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06d7d761_d47f_4aa5_b931_46f910b65597.slice/crio-e397a0c944576f2074bc739223c5335638af23c6d034adc10e713a89abf7cf3f WatchSource:0}: Error finding container e397a0c944576f2074bc739223c5335638af23c6d034adc10e713a89abf7cf3f: Status 404 returned error can't find the container with id e397a0c944576f2074bc739223c5335638af23c6d034adc10e713a89abf7cf3f Oct 30 13:10:42.205658 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:42.205619 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" event={"ID":"06d7d761-d47f-4aa5-b931-46f910b65597","Type":"ContainerStarted","Data":"a65ea87c5d1877ced1ecac3014756daedd938caf6c1628f818ff8573deac55ba"} Oct 30 13:10:42.206484 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:42.205666 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" event={"ID":"06d7d761-d47f-4aa5-b931-46f910b65597","Type":"ContainerStarted","Data":"e397a0c944576f2074bc739223c5335638af23c6d034adc10e713a89abf7cf3f"} Oct 30 13:10:42.260821 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:42.260765 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" podStartSLOduration=2.26074868 podStartE2EDuration="2.26074868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:10:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:10:42.25866014 +0000 UTC m=+1141.284301389" watchObservedRunningTime="2026-10-30 13:10:42.26074868 +0000 UTC m=+1141.286389929" Oct 30 13:10:42.852274 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:42.852241 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:44.632643 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:44.632605 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t"] Oct 30 13:10:44.719254 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:44.719219 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb"] Oct 30 13:10:44.883198 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:10:44.883101 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:10:46.843471 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:46.843434 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh"] Oct 30 13:10:50.851662 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:50.851626 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:50.863840 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:50.863810 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:51.284024 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:51.283934 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:10:51.844715 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:51.844654 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh" podUID="5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6" containerName="payload-processing" containerID="cri-o://cc89e1cf5f89ec71579e0e1ff3f3c8958085af8212a45fecb90fc80aa3a3d665" gracePeriod=25 Oct 30 13:10:52.326500 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:52.326464 5029 generic.go:356] "Generic (PLEG): container finished" podID="5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6" containerID="cc89e1cf5f89ec71579e0e1ff3f3c8958085af8212a45fecb90fc80aa3a3d665" exitCode=0 Oct 30 13:10:52.327912 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:52.327883 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh" event={"ID":"5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6","Type":"ContainerDied","Data":"cc89e1cf5f89ec71579e0e1ff3f3c8958085af8212a45fecb90fc80aa3a3d665"} Oct 30 13:10:52.381951 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:52.381912 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh" Oct 30 13:10:52.486105 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:52.486000 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6-plugins-config-volume\") pod \"5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6\" (UID: \"5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6\") " Oct 30 13:10:52.487857 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:52.487823 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6-plugins-config-volume" pod "5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6" (UID: "5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:10:52.490039 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:52.489996 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6-kube-api-access-7bmbp\" (UniqueName: \"kubernetes.io/projected/5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6-kube-api-access-7bmbp\") pod \"5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6\" (UID: \"5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6\") " Oct 30 13:10:52.493671 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:52.493653 5029 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6-plugins-config-volume\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:10:52.495461 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:52.495438 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6-kube-api-access-7bmbp" pod "5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6" (UID: "5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6"). InnerVolumeSpecName "kube-api-access-7bmbp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:10:52.595369 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:52.595330 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bmbp\" (UniqueName: \"kubernetes.io/projected/5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6-kube-api-access-7bmbp\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:10:53.340614 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:53.340581 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh" Oct 30 13:10:53.341101 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:53.340612 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh" event={"ID":"5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6","Type":"ContainerDied","Data":"4b2b888309678162ac9b2f6cec7122ffa9524b4f9b6ad261f8e564290a3f0b70"} Oct 30 13:10:53.341101 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:53.340666 5029 scope.go:122] "RemoveContainer" containerID="cc89e1cf5f89ec71579e0e1ff3f3c8958085af8212a45fecb90fc80aa3a3d665" Oct 30 13:10:53.400804 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:53.400729 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh"] Oct 30 13:10:53.412293 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:53.412267 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-253f7c37-545fc54778-bnjdh"] Oct 30 13:10:55.091921 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:55.091890 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6" path="/var/lib/kubelet/pods/5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6/volumes" Oct 30 13:10:55.460729 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:55.460610 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr"] Oct 30 13:10:55.460909 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:55.460864 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" podUID="3447ed0f-ef06-4a75-8f49-67c043cf08cb" containerName="istio-proxy" containerID="cri-o://0d9a68502d58a92011d992a15466590baa8b5b4d0bc9f0348c1effb1af6d1f95" gracePeriod=30 Oct 30 13:10:59.654738 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:59.654686 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" podUID="607535ec-5cbf-4b96-a053-6519ddc2302b" containerName="main" containerID="cri-o://4c465c0a709deeb88d129f0396969c77447803a4bd24c3fbe714b8a04afa272a" gracePeriod=45 Oct 30 13:10:59.741235 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:10:59.741200 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" podUID="99e695e7-e73c-429c-87ca-39ee740917c1" containerName="main" containerID="cri-o://af2ad9aebf66050565eaea52da7b746a5815002ba3920d9e83536c9f5eb3f236" gracePeriod=45 Oct 30 13:11:00.092946 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.092522 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:11:00.135121 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.135056 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:11:00.153457 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.153426 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-dshm\") pod \"607535ec-5cbf-4b96-a053-6519ddc2302b\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " Oct 30 13:11:00.153601 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.153520 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/607535ec-5cbf-4b96-a053-6519ddc2302b-tls-certs\" (UniqueName: \"kubernetes.io/secret/607535ec-5cbf-4b96-a053-6519ddc2302b-tls-certs\") pod \"607535ec-5cbf-4b96-a053-6519ddc2302b\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " Oct 30 13:11:00.153601 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.153565 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/607535ec-5cbf-4b96-a053-6519ddc2302b-kube-api-access-pnshr\" (UniqueName: \"kubernetes.io/projected/607535ec-5cbf-4b96-a053-6519ddc2302b-kube-api-access-pnshr\") pod \"607535ec-5cbf-4b96-a053-6519ddc2302b\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " Oct 30 13:11:00.153673 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.153608 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-home\" (UniqueName: \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-home\") pod \"607535ec-5cbf-4b96-a053-6519ddc2302b\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " Oct 30 13:11:00.153766 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.153755 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-tmp-dir\") pod \"607535ec-5cbf-4b96-a053-6519ddc2302b\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " Oct 30 13:11:00.153826 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.153796 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-model-cache\") pod \"607535ec-5cbf-4b96-a053-6519ddc2302b\" (UID: \"607535ec-5cbf-4b96-a053-6519ddc2302b\") " Oct 30 13:11:00.155608 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.155584 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-home" pod "607535ec-5cbf-4b96-a053-6519ddc2302b" (UID: "607535ec-5cbf-4b96-a053-6519ddc2302b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:00.157449 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.157170 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-model-cache" pod "607535ec-5cbf-4b96-a053-6519ddc2302b" (UID: "607535ec-5cbf-4b96-a053-6519ddc2302b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:00.157589 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.157451 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-dshm" pod "607535ec-5cbf-4b96-a053-6519ddc2302b" (UID: "607535ec-5cbf-4b96-a053-6519ddc2302b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:00.157710 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.157683 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-tmp-dir" pod "607535ec-5cbf-4b96-a053-6519ddc2302b" (UID: "607535ec-5cbf-4b96-a053-6519ddc2302b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:00.158395 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.158364 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/607535ec-5cbf-4b96-a053-6519ddc2302b-tls-certs" pod "607535ec-5cbf-4b96-a053-6519ddc2302b" (UID: "607535ec-5cbf-4b96-a053-6519ddc2302b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:11:00.159423 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.159397 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/607535ec-5cbf-4b96-a053-6519ddc2302b-kube-api-access-pnshr" pod "607535ec-5cbf-4b96-a053-6519ddc2302b" (UID: "607535ec-5cbf-4b96-a053-6519ddc2302b"). InnerVolumeSpecName "kube-api-access-pnshr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:11:00.254922 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.254829 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-dshm\") pod \"99e695e7-e73c-429c-87ca-39ee740917c1\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " Oct 30 13:11:00.254922 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.254917 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/99e695e7-e73c-429c-87ca-39ee740917c1-tls-certs\" (UniqueName: \"kubernetes.io/secret/99e695e7-e73c-429c-87ca-39ee740917c1-tls-certs\") pod \"99e695e7-e73c-429c-87ca-39ee740917c1\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " Oct 30 13:11:00.255220 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.255120 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-home\" (UniqueName: \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-home\") pod \"99e695e7-e73c-429c-87ca-39ee740917c1\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " Oct 30 13:11:00.255220 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.255191 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99e695e7-e73c-429c-87ca-39ee740917c1-kube-api-access-n5jl5\" (UniqueName: \"kubernetes.io/projected/99e695e7-e73c-429c-87ca-39ee740917c1-kube-api-access-n5jl5\") pod \"99e695e7-e73c-429c-87ca-39ee740917c1\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " Oct 30 13:11:00.255337 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.255232 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-tmp-dir\") pod \"99e695e7-e73c-429c-87ca-39ee740917c1\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " Oct 30 13:11:00.255337 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.255293 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-model-cache\") pod \"99e695e7-e73c-429c-87ca-39ee740917c1\" (UID: \"99e695e7-e73c-429c-87ca-39ee740917c1\") " Oct 30 13:11:00.255557 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.255526 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-home" pod "99e695e7-e73c-429c-87ca-39ee740917c1" (UID: "99e695e7-e73c-429c-87ca-39ee740917c1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:00.255771 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.255749 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-model-cache" pod "99e695e7-e73c-429c-87ca-39ee740917c1" (UID: "99e695e7-e73c-429c-87ca-39ee740917c1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:00.256172 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.256146 5029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/607535ec-5cbf-4b96-a053-6519ddc2302b-tls-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:00.256282 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.256174 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnshr\" (UniqueName: \"kubernetes.io/projected/607535ec-5cbf-4b96-a053-6519ddc2302b-kube-api-access-pnshr\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:00.256282 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.256190 5029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-home\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:00.256282 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.256203 5029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-home\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:00.256282 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.256215 5029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-model-cache\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:00.256282 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.256227 5029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-tmp-dir\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:00.256282 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.256240 5029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-model-cache\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:00.256282 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.256251 5029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/607535ec-5cbf-4b96-a053-6519ddc2302b-dshm\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:00.256671 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.256462 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-tmp-dir" pod "99e695e7-e73c-429c-87ca-39ee740917c1" (UID: "99e695e7-e73c-429c-87ca-39ee740917c1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:00.257476 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.257454 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-dshm" pod "99e695e7-e73c-429c-87ca-39ee740917c1" (UID: "99e695e7-e73c-429c-87ca-39ee740917c1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:00.258856 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.258830 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99e695e7-e73c-429c-87ca-39ee740917c1-tls-certs" pod "99e695e7-e73c-429c-87ca-39ee740917c1" (UID: "99e695e7-e73c-429c-87ca-39ee740917c1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:11:00.258856 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.258845 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99e695e7-e73c-429c-87ca-39ee740917c1-kube-api-access-n5jl5" pod "99e695e7-e73c-429c-87ca-39ee740917c1" (UID: "99e695e7-e73c-429c-87ca-39ee740917c1"). InnerVolumeSpecName "kube-api-access-n5jl5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:11:00.357690 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.357655 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5jl5\" (UniqueName: \"kubernetes.io/projected/99e695e7-e73c-429c-87ca-39ee740917c1-kube-api-access-n5jl5\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:00.357856 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.357687 5029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-tmp-dir\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:00.357856 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.357712 5029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99e695e7-e73c-429c-87ca-39ee740917c1-dshm\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:00.357856 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.357721 5029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99e695e7-e73c-429c-87ca-39ee740917c1-tls-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:00.390247 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.390213 5029 generic.go:356] "Generic (PLEG): container finished" podID="99e695e7-e73c-429c-87ca-39ee740917c1" containerID="af2ad9aebf66050565eaea52da7b746a5815002ba3920d9e83536c9f5eb3f236" exitCode=0 Oct 30 13:11:00.390385 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.390257 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" Oct 30 13:11:00.390385 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.390297 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" event={"ID":"99e695e7-e73c-429c-87ca-39ee740917c1","Type":"ContainerDied","Data":"af2ad9aebf66050565eaea52da7b746a5815002ba3920d9e83536c9f5eb3f236"} Oct 30 13:11:00.390385 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.390336 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" event={"ID":"99e695e7-e73c-429c-87ca-39ee740917c1","Type":"ContainerDied","Data":"d44a3e56f225a374066d534932daa5a50855dd79518453f91316dcb0cbb36e33"} Oct 30 13:11:00.390385 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.390354 5029 scope.go:122] "RemoveContainer" containerID="af2ad9aebf66050565eaea52da7b746a5815002ba3920d9e83536c9f5eb3f236" Oct 30 13:11:00.392012 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.391983 5029 generic.go:356] "Generic (PLEG): container finished" podID="607535ec-5cbf-4b96-a053-6519ddc2302b" containerID="4c465c0a709deeb88d129f0396969c77447803a4bd24c3fbe714b8a04afa272a" exitCode=0 Oct 30 13:11:00.392012 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.392010 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" event={"ID":"607535ec-5cbf-4b96-a053-6519ddc2302b","Type":"ContainerDied","Data":"4c465c0a709deeb88d129f0396969c77447803a4bd24c3fbe714b8a04afa272a"} Oct 30 13:11:00.392176 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.392033 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" event={"ID":"607535ec-5cbf-4b96-a053-6519ddc2302b","Type":"ContainerDied","Data":"abe6c182468bed9ad11f8f69e43ad0e7d6fa8e877c75e12bce1025e388d1c147"} Oct 30 13:11:00.392176 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.392119 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t" Oct 30 13:11:00.401041 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.401020 5029 scope.go:122] "RemoveContainer" containerID="af2ad9aebf66050565eaea52da7b746a5815002ba3920d9e83536c9f5eb3f236" Oct 30 13:11:00.401377 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:11:00.401351 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af2ad9aebf66050565eaea52da7b746a5815002ba3920d9e83536c9f5eb3f236\": container with ID starting with af2ad9aebf66050565eaea52da7b746a5815002ba3920d9e83536c9f5eb3f236 not found: ID does not exist" containerID="af2ad9aebf66050565eaea52da7b746a5815002ba3920d9e83536c9f5eb3f236" Oct 30 13:11:00.401471 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.401389 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af2ad9aebf66050565eaea52da7b746a5815002ba3920d9e83536c9f5eb3f236"} err="failed to get container status \"af2ad9aebf66050565eaea52da7b746a5815002ba3920d9e83536c9f5eb3f236\": rpc error: code = NotFound desc = could not find container \"af2ad9aebf66050565eaea52da7b746a5815002ba3920d9e83536c9f5eb3f236\": container with ID starting with af2ad9aebf66050565eaea52da7b746a5815002ba3920d9e83536c9f5eb3f236 not found: ID does not exist" Oct 30 13:11:00.401471 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.401409 5029 scope.go:122] "RemoveContainer" containerID="4c465c0a709deeb88d129f0396969c77447803a4bd24c3fbe714b8a04afa272a" Oct 30 13:11:00.410133 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.410116 5029 scope.go:122] "RemoveContainer" containerID="4c465c0a709deeb88d129f0396969c77447803a4bd24c3fbe714b8a04afa272a" Oct 30 13:11:00.410364 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:11:00.410345 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c465c0a709deeb88d129f0396969c77447803a4bd24c3fbe714b8a04afa272a\": container with ID starting with 4c465c0a709deeb88d129f0396969c77447803a4bd24c3fbe714b8a04afa272a not found: ID does not exist" containerID="4c465c0a709deeb88d129f0396969c77447803a4bd24c3fbe714b8a04afa272a" Oct 30 13:11:00.410422 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.410371 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c465c0a709deeb88d129f0396969c77447803a4bd24c3fbe714b8a04afa272a"} err="failed to get container status \"4c465c0a709deeb88d129f0396969c77447803a4bd24c3fbe714b8a04afa272a\": rpc error: code = NotFound desc = could not find container \"4c465c0a709deeb88d129f0396969c77447803a4bd24c3fbe714b8a04afa272a\": container with ID starting with 4c465c0a709deeb88d129f0396969c77447803a4bd24c3fbe714b8a04afa272a not found: ID does not exist" Oct 30 13:11:00.442974 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.442942 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t"] Oct 30 13:11:00.449185 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.449162 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-42ee9d/distinct-2-w0-42ee9d-kserve-8d8859d4c-fjd8t"] Oct 30 13:11:00.484114 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.484090 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb"] Oct 30 13:11:00.490272 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.490248 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb"] Oct 30 13:11:00.546785 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.546709 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc"] Oct 30 13:11:00.547639 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.547619 5029 cpu_manager.go:414] "containerMap: removing container" podUID="607535ec-5cbf-4b96-a053-6519ddc2302b" containerName="main" Oct 30 13:11:00.547705 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.547643 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="607535ec-5cbf-4b96-a053-6519ddc2302b" containerName="main" Oct 30 13:11:00.547705 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.547665 5029 cpu_manager.go:414] "containerMap: removing container" podUID="99e695e7-e73c-429c-87ca-39ee740917c1" containerName="main" Oct 30 13:11:00.547705 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.547673 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99e695e7-e73c-429c-87ca-39ee740917c1" containerName="main" Oct 30 13:11:00.547798 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.547727 5029 cpu_manager.go:414] "containerMap: removing container" podUID="5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6" containerName="payload-processing" Oct 30 13:11:00.547798 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.547736 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6" containerName="payload-processing" Oct 30 13:11:00.547903 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.547890 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="607535ec-5cbf-4b96-a053-6519ddc2302b" containerName="main" Oct 30 13:11:00.547938 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.547909 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="99e695e7-e73c-429c-87ca-39ee740917c1" containerName="main" Oct 30 13:11:00.547938 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.547921 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d7c6015-cbd6-4b2c-ace9-dd9a73a001b6" containerName="payload-processing" Oct 30 13:11:00.550992 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.550972 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.570613 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.570585 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Oct 30 13:11:00.571434 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.571415 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-fqrgx\"" Oct 30 13:11:00.571522 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.571492 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Oct 30 13:11:00.571579 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.571532 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Oct 30 13:11:00.573542 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.573514 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc"] Oct 30 13:11:00.662543 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.662510 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-m2qwc\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.662975 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.662559 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b89f6e6-34f4-4888-bea4-26ed65d618e2-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-m2qwc\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.662975 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.662657 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-m2qwc\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.662975 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.662768 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqv2p\" (UniqueName: \"kubernetes.io/projected/9b89f6e6-34f4-4888-bea4-26ed65d618e2-kube-api-access-hqv2p\") pod \"istiod-openshift-gateway-7ddd9cfdc8-m2qwc\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.662975 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.662883 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-m2qwc\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.662975 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.662966 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9b89f6e6-34f4-4888-bea4-26ed65d618e2-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-m2qwc\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.663265 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.662992 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-m2qwc\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.764576 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.764547 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-m2qwc\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.764734 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.764634 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqv2p\" (UniqueName: \"kubernetes.io/projected/9b89f6e6-34f4-4888-bea4-26ed65d618e2-kube-api-access-hqv2p\") pod \"istiod-openshift-gateway-7ddd9cfdc8-m2qwc\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.764734 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.764714 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-m2qwc\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.764846 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.764772 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9b89f6e6-34f4-4888-bea4-26ed65d618e2-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-m2qwc\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.764846 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.764798 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-m2qwc\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.764940 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.764862 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-m2qwc\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.764940 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.764891 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b89f6e6-34f4-4888-bea4-26ed65d618e2-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-m2qwc\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.765508 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.765475 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-m2qwc\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.767474 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.767444 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-m2qwc\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.767657 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.767635 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-m2qwc\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.767744 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.767665 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b89f6e6-34f4-4888-bea4-26ed65d618e2-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-m2qwc\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.768580 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.768558 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9b89f6e6-34f4-4888-bea4-26ed65d618e2-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-m2qwc\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.790113 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.790060 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-m2qwc\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.790235 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.790088 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqv2p\" (UniqueName: \"kubernetes.io/projected/9b89f6e6-34f4-4888-bea4-26ed65d618e2-kube-api-access-hqv2p\") pod \"istiod-openshift-gateway-7ddd9cfdc8-m2qwc\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.801718 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.801661 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:11:00.867523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.862738 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:00.867523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.865509 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-credential-socket\") pod \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " Oct 30 13:11:00.867523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.865645 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-envoy\") pod \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " Oct 30 13:11:00.867523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.865707 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-data\") pod \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " Oct 30 13:11:00.867523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.865764 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-token\" (UniqueName: \"kubernetes.io/projected/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-token\") pod \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " Oct 30 13:11:00.867523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.865822 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-podinfo\") pod \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " Oct 30 13:11:00.867523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.865823 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-credential-socket" pod "3447ed0f-ef06-4a75-8f49-67c043cf08cb" (UID: "3447ed0f-ef06-4a75-8f49-67c043cf08cb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:00.867523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.865856 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3447ed0f-ef06-4a75-8f49-67c043cf08cb-kube-api-access-ptvf5\" (UniqueName: \"kubernetes.io/projected/3447ed0f-ef06-4a75-8f49-67c043cf08cb-kube-api-access-ptvf5\") pod \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " Oct 30 13:11:00.867523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.865899 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-workload-certs\") pod \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " Oct 30 13:11:00.867523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.865954 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-workload-socket\") pod \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " Oct 30 13:11:00.867523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.866018 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-data" pod "3447ed0f-ef06-4a75-8f49-67c043cf08cb" (UID: "3447ed0f-ef06-4a75-8f49-67c043cf08cb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:00.867523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.866040 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istiod-ca-cert\") pod \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\" (UID: \"3447ed0f-ef06-4a75-8f49-67c043cf08cb\") " Oct 30 13:11:00.867523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.866690 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-credential-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:00.867523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.866712 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-data\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:00.881445 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.868767 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istiod-ca-cert" pod "3447ed0f-ef06-4a75-8f49-67c043cf08cb" (UID: "3447ed0f-ef06-4a75-8f49-67c043cf08cb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:11:00.881445 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.869030 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-workload-certs" pod "3447ed0f-ef06-4a75-8f49-67c043cf08cb" (UID: "3447ed0f-ef06-4a75-8f49-67c043cf08cb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:00.881445 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.869226 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-workload-socket" pod "3447ed0f-ef06-4a75-8f49-67c043cf08cb" (UID: "3447ed0f-ef06-4a75-8f49-67c043cf08cb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:00.881445 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.869970 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3447ed0f-ef06-4a75-8f49-67c043cf08cb-kube-api-access-ptvf5" pod "3447ed0f-ef06-4a75-8f49-67c043cf08cb" (UID: "3447ed0f-ef06-4a75-8f49-67c043cf08cb"). InnerVolumeSpecName "kube-api-access-ptvf5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:11:00.881445 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.876401 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-token" pod "3447ed0f-ef06-4a75-8f49-67c043cf08cb" (UID: "3447ed0f-ef06-4a75-8f49-67c043cf08cb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:11:00.881445 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.876689 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-podinfo" pod "3447ed0f-ef06-4a75-8f49-67c043cf08cb" (UID: "3447ed0f-ef06-4a75-8f49-67c043cf08cb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 13:11:00.893227 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.893190 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-envoy" pod "3447ed0f-ef06-4a75-8f49-67c043cf08cb" (UID: "3447ed0f-ef06-4a75-8f49-67c043cf08cb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:00.970633 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.969334 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptvf5\" (UniqueName: \"kubernetes.io/projected/3447ed0f-ef06-4a75-8f49-67c043cf08cb-kube-api-access-ptvf5\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:00.970633 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.969373 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-workload-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:00.970633 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.969390 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-workload-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:00.970633 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.969404 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istiod-ca-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:00.970633 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.969456 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-envoy\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:00.970633 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.969468 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:00.970633 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:00.969478 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3447ed0f-ef06-4a75-8f49-67c043cf08cb-istio-podinfo\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:01.074308 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:01.074269 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-42ee9d/simulator-w0-42ee9d-kserve-575877dfdd-rdcnb" podUID="99e695e7-e73c-429c-87ca-39ee740917c1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:11:01.092235 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:01.092206 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="607535ec-5cbf-4b96-a053-6519ddc2302b" path="/var/lib/kubelet/pods/607535ec-5cbf-4b96-a053-6519ddc2302b/volumes" Oct 30 13:11:01.092673 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:01.092657 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="99e695e7-e73c-429c-87ca-39ee740917c1" path="/var/lib/kubelet/pods/99e695e7-e73c-429c-87ca-39ee740917c1/volumes" Oct 30 13:11:01.210636 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:01.210615 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc"] Oct 30 13:11:01.213282 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:11:01.213255 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b89f6e6_34f4_4888_bea4_26ed65d618e2.slice/crio-0dec5a6af3c61275a912c50760b03ec16313095261b06f040e34d6d078516bc1 WatchSource:0}: Error finding container 0dec5a6af3c61275a912c50760b03ec16313095261b06f040e34d6d078516bc1: Status 404 returned error can't find the container with id 0dec5a6af3c61275a912c50760b03ec16313095261b06f040e34d6d078516bc1 Oct 30 13:11:01.399801 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:01.399716 5029 generic.go:356] "Generic (PLEG): container finished" podID="3447ed0f-ef06-4a75-8f49-67c043cf08cb" containerID="0d9a68502d58a92011d992a15466590baa8b5b4d0bc9f0348c1effb1af6d1f95" exitCode=0 Oct 30 13:11:01.399801 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:01.399765 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" Oct 30 13:11:01.399801 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:01.399782 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" event={"ID":"3447ed0f-ef06-4a75-8f49-67c043cf08cb","Type":"ContainerDied","Data":"0d9a68502d58a92011d992a15466590baa8b5b4d0bc9f0348c1effb1af6d1f95"} Oct 30 13:11:01.400103 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:01.399839 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr" event={"ID":"3447ed0f-ef06-4a75-8f49-67c043cf08cb","Type":"ContainerDied","Data":"32afcccb6b11ca37b3f621b3973b1e535fb3b17effef110789763b179fdebaf2"} Oct 30 13:11:01.400103 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:01.399871 5029 scope.go:122] "RemoveContainer" containerID="0d9a68502d58a92011d992a15466590baa8b5b4d0bc9f0348c1effb1af6d1f95" Oct 30 13:11:01.402646 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:01.402607 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" event={"ID":"9b89f6e6-34f4-4888-bea4-26ed65d618e2","Type":"ContainerStarted","Data":"0dec5a6af3c61275a912c50760b03ec16313095261b06f040e34d6d078516bc1"} Oct 30 13:11:01.410053 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:01.410035 5029 scope.go:122] "RemoveContainer" containerID="0d9a68502d58a92011d992a15466590baa8b5b4d0bc9f0348c1effb1af6d1f95" Oct 30 13:11:01.410367 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:11:01.410344 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d9a68502d58a92011d992a15466590baa8b5b4d0bc9f0348c1effb1af6d1f95\": container with ID starting with 0d9a68502d58a92011d992a15466590baa8b5b4d0bc9f0348c1effb1af6d1f95 not found: ID does not exist" containerID="0d9a68502d58a92011d992a15466590baa8b5b4d0bc9f0348c1effb1af6d1f95" Oct 30 13:11:01.410431 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:01.410377 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d9a68502d58a92011d992a15466590baa8b5b4d0bc9f0348c1effb1af6d1f95"} err="failed to get container status \"0d9a68502d58a92011d992a15466590baa8b5b4d0bc9f0348c1effb1af6d1f95\": rpc error: code = NotFound desc = could not find container \"0d9a68502d58a92011d992a15466590baa8b5b4d0bc9f0348c1effb1af6d1f95\": container with ID starting with 0d9a68502d58a92011d992a15466590baa8b5b4d0bc9f0348c1effb1af6d1f95 not found: ID does not exist" Oct 30 13:11:01.454319 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:01.454293 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr"] Oct 30 13:11:01.465862 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:01.465836 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-h4fjr"] Oct 30 13:11:03.093246 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:03.093213 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3447ed0f-ef06-4a75-8f49-67c043cf08cb" path="/var/lib/kubelet/pods/3447ed0f-ef06-4a75-8f49-67c043cf08cb/volumes" Oct 30 13:11:03.716881 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:03.716841 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:11:03.717021 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:03.716924 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:11:04.432515 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:04.432463 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" event={"ID":"9b89f6e6-34f4-4888-bea4-26ed65d618e2","Type":"ContainerStarted","Data":"14a69174c80b43eedd3cf2d47e9002315412cb3c4399d84aa7fd71869445e64a"} Oct 30 13:11:04.432957 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:04.432540 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:04.434571 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:04.434545 5029 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-m2qwc container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 13:11:04.434699 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:04.434598 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" podUID="9b89f6e6-34f4-4888-bea4-26ed65d618e2" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 13:11:04.501467 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:04.501406 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" podStartSLOduration=2.001128802 podStartE2EDuration="4.50138797s" totalImagesPullingTime="2.500259168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:11:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:11:04.488516639 +0000 UTC m=+1163.514157912" watchObservedRunningTime="2026-10-30 13:11:04.50138797 +0000 UTC m=+1163.527029218" Oct 30 13:11:05.440161 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:05.440133 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:11:07.179607 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:07.179569 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r"] Oct 30 13:11:07.194871 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:07.194833 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf"] Oct 30 13:11:14.880812 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:11:14.880773 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:11:22.206382 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.206344 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" podUID="9af7cbdb-5aee-4f11-9446-fceacd3fb135" containerName="main" containerID="cri-o://f7b79dab050cc1d07509dee3eafa8236eaef2eb354f2bd1e37f55e68208257e0" gracePeriod=45 Oct 30 13:11:22.218685 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.218647 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" podUID="935c23b1-953a-40cc-ad0c-bec2ef61afba" containerName="main" containerID="cri-o://1edced17b2f6ec44447f1f829686afd38fdd3b9f2dca386e345b7c03c5fd57ed" gracePeriod=45 Oct 30 13:11:22.524083 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.511858 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:11:22.542904 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.542880 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:11:22.554385 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.554357 5029 generic.go:356] "Generic (PLEG): container finished" podID="935c23b1-953a-40cc-ad0c-bec2ef61afba" containerID="1edced17b2f6ec44447f1f829686afd38fdd3b9f2dca386e345b7c03c5fd57ed" exitCode=0 Oct 30 13:11:22.554521 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.554411 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" Oct 30 13:11:22.554521 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.554437 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" event={"ID":"935c23b1-953a-40cc-ad0c-bec2ef61afba","Type":"ContainerDied","Data":"1edced17b2f6ec44447f1f829686afd38fdd3b9f2dca386e345b7c03c5fd57ed"} Oct 30 13:11:22.554521 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.554475 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf" event={"ID":"935c23b1-953a-40cc-ad0c-bec2ef61afba","Type":"ContainerDied","Data":"6703f522eef94489618e043a1353aaac80d21aa41e151c138af63683837cdd54"} Oct 30 13:11:22.554521 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.554488 5029 scope.go:122] "RemoveContainer" containerID="1edced17b2f6ec44447f1f829686afd38fdd3b9f2dca386e345b7c03c5fd57ed" Oct 30 13:11:22.555725 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.555706 5029 generic.go:356] "Generic (PLEG): container finished" podID="9af7cbdb-5aee-4f11-9446-fceacd3fb135" containerID="f7b79dab050cc1d07509dee3eafa8236eaef2eb354f2bd1e37f55e68208257e0" exitCode=0 Oct 30 13:11:22.555879 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.555865 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" event={"ID":"9af7cbdb-5aee-4f11-9446-fceacd3fb135","Type":"ContainerDied","Data":"f7b79dab050cc1d07509dee3eafa8236eaef2eb354f2bd1e37f55e68208257e0"} Oct 30 13:11:22.555939 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.555892 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" event={"ID":"9af7cbdb-5aee-4f11-9446-fceacd3fb135","Type":"ContainerDied","Data":"7f6c05b181f68cdc85cd65e6440505182271e5fc3bd5d642a6b3ded7739191b8"} Oct 30 13:11:22.555993 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.555941 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" Oct 30 13:11:22.565606 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.565580 5029 scope.go:122] "RemoveContainer" containerID="1edced17b2f6ec44447f1f829686afd38fdd3b9f2dca386e345b7c03c5fd57ed" Oct 30 13:11:22.565992 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:11:22.565964 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1edced17b2f6ec44447f1f829686afd38fdd3b9f2dca386e345b7c03c5fd57ed\": container with ID starting with 1edced17b2f6ec44447f1f829686afd38fdd3b9f2dca386e345b7c03c5fd57ed not found: ID does not exist" containerID="1edced17b2f6ec44447f1f829686afd38fdd3b9f2dca386e345b7c03c5fd57ed" Oct 30 13:11:22.566134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.566005 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1edced17b2f6ec44447f1f829686afd38fdd3b9f2dca386e345b7c03c5fd57ed"} err="failed to get container status \"1edced17b2f6ec44447f1f829686afd38fdd3b9f2dca386e345b7c03c5fd57ed\": rpc error: code = NotFound desc = could not find container \"1edced17b2f6ec44447f1f829686afd38fdd3b9f2dca386e345b7c03c5fd57ed\": container with ID starting with 1edced17b2f6ec44447f1f829686afd38fdd3b9f2dca386e345b7c03c5fd57ed not found: ID does not exist" Oct 30 13:11:22.566134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.566031 5029 scope.go:122] "RemoveContainer" containerID="f7b79dab050cc1d07509dee3eafa8236eaef2eb354f2bd1e37f55e68208257e0" Oct 30 13:11:22.583564 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.583544 5029 scope.go:122] "RemoveContainer" containerID="f7b79dab050cc1d07509dee3eafa8236eaef2eb354f2bd1e37f55e68208257e0" Oct 30 13:11:22.584053 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:11:22.584029 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7b79dab050cc1d07509dee3eafa8236eaef2eb354f2bd1e37f55e68208257e0\": container with ID starting with f7b79dab050cc1d07509dee3eafa8236eaef2eb354f2bd1e37f55e68208257e0 not found: ID does not exist" containerID="f7b79dab050cc1d07509dee3eafa8236eaef2eb354f2bd1e37f55e68208257e0" Oct 30 13:11:22.584253 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.584228 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7b79dab050cc1d07509dee3eafa8236eaef2eb354f2bd1e37f55e68208257e0"} err="failed to get container status \"f7b79dab050cc1d07509dee3eafa8236eaef2eb354f2bd1e37f55e68208257e0\": rpc error: code = NotFound desc = could not find container \"f7b79dab050cc1d07509dee3eafa8236eaef2eb354f2bd1e37f55e68208257e0\": container with ID starting with f7b79dab050cc1d07509dee3eafa8236eaef2eb354f2bd1e37f55e68208257e0 not found: ID does not exist" Oct 30 13:11:22.617131 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.617061 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-tmp-dir\") pod \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " Oct 30 13:11:22.617416 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.617378 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-tmp-dir" pod "9af7cbdb-5aee-4f11-9446-fceacd3fb135" (UID: "9af7cbdb-5aee-4f11-9446-fceacd3fb135"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:22.617494 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.617387 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9af7cbdb-5aee-4f11-9446-fceacd3fb135-tls-certs\" (UniqueName: \"kubernetes.io/secret/9af7cbdb-5aee-4f11-9446-fceacd3fb135-tls-certs\") pod \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " Oct 30 13:11:22.617805 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.617783 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-dshm\") pod \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " Oct 30 13:11:22.618253 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.618231 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-home\" (UniqueName: \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-home\") pod \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " Oct 30 13:11:22.618542 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.618498 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-home" pod "9af7cbdb-5aee-4f11-9446-fceacd3fb135" (UID: "9af7cbdb-5aee-4f11-9446-fceacd3fb135"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:22.630278 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.630233 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-dshm" pod "9af7cbdb-5aee-4f11-9446-fceacd3fb135" (UID: "9af7cbdb-5aee-4f11-9446-fceacd3fb135"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:22.633723 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.633698 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9af7cbdb-5aee-4f11-9446-fceacd3fb135-kube-api-access-6qzfp\" (UniqueName: \"kubernetes.io/projected/9af7cbdb-5aee-4f11-9446-fceacd3fb135-kube-api-access-6qzfp\") pod \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " Oct 30 13:11:22.634324 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.634306 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-model-cache\") pod \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\" (UID: \"9af7cbdb-5aee-4f11-9446-fceacd3fb135\") " Oct 30 13:11:22.634779 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.634660 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-model-cache" pod "9af7cbdb-5aee-4f11-9446-fceacd3fb135" (UID: "9af7cbdb-5aee-4f11-9446-fceacd3fb135"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:22.635597 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.635577 5029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-dshm\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:22.635725 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.635711 5029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-home\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:22.635819 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.635806 5029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-model-cache\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:22.635910 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.635899 5029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9af7cbdb-5aee-4f11-9446-fceacd3fb135-tmp-dir\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:22.638434 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.638409 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9af7cbdb-5aee-4f11-9446-fceacd3fb135-tls-certs" pod "9af7cbdb-5aee-4f11-9446-fceacd3fb135" (UID: "9af7cbdb-5aee-4f11-9446-fceacd3fb135"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:11:22.641439 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.641397 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9af7cbdb-5aee-4f11-9446-fceacd3fb135-kube-api-access-6qzfp" pod "9af7cbdb-5aee-4f11-9446-fceacd3fb135" (UID: "9af7cbdb-5aee-4f11-9446-fceacd3fb135"). InnerVolumeSpecName "kube-api-access-6qzfp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:11:22.737303 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.737216 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/935c23b1-953a-40cc-ad0c-bec2ef61afba-tls-certs\" (UniqueName: \"kubernetes.io/secret/935c23b1-953a-40cc-ad0c-bec2ef61afba-tls-certs\") pod \"935c23b1-953a-40cc-ad0c-bec2ef61afba\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " Oct 30 13:11:22.737303 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.737270 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-tmp-dir\") pod \"935c23b1-953a-40cc-ad0c-bec2ef61afba\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " Oct 30 13:11:22.737538 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.737314 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-model-cache\") pod \"935c23b1-953a-40cc-ad0c-bec2ef61afba\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " Oct 30 13:11:22.737538 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.737515 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-home\" (UniqueName: \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-home\") pod \"935c23b1-953a-40cc-ad0c-bec2ef61afba\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " Oct 30 13:11:22.737653 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.737558 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/935c23b1-953a-40cc-ad0c-bec2ef61afba-kube-api-access-9pfr8\" (UniqueName: \"kubernetes.io/projected/935c23b1-953a-40cc-ad0c-bec2ef61afba-kube-api-access-9pfr8\") pod \"935c23b1-953a-40cc-ad0c-bec2ef61afba\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " Oct 30 13:11:22.737653 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.737595 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-tmp-dir" pod "935c23b1-953a-40cc-ad0c-bec2ef61afba" (UID: "935c23b1-953a-40cc-ad0c-bec2ef61afba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:22.737770 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.737629 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-dshm\" (UniqueName: \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-dshm\") pod \"935c23b1-953a-40cc-ad0c-bec2ef61afba\" (UID: \"935c23b1-953a-40cc-ad0c-bec2ef61afba\") " Oct 30 13:11:22.737828 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.737647 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-model-cache" pod "935c23b1-953a-40cc-ad0c-bec2ef61afba" (UID: "935c23b1-953a-40cc-ad0c-bec2ef61afba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:22.737828 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.737758 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-home" pod "935c23b1-953a-40cc-ad0c-bec2ef61afba" (UID: "935c23b1-953a-40cc-ad0c-bec2ef61afba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:22.738245 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.738223 5029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-home\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:22.738245 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.738246 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6qzfp\" (UniqueName: \"kubernetes.io/projected/9af7cbdb-5aee-4f11-9446-fceacd3fb135-kube-api-access-6qzfp\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:22.738417 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.738256 5029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-tmp-dir\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:22.738417 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.738266 5029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9af7cbdb-5aee-4f11-9446-fceacd3fb135-tls-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:22.738417 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.738274 5029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-model-cache\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:22.739722 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.739695 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/935c23b1-953a-40cc-ad0c-bec2ef61afba-kube-api-access-9pfr8" pod "935c23b1-953a-40cc-ad0c-bec2ef61afba" (UID: "935c23b1-953a-40cc-ad0c-bec2ef61afba"). InnerVolumeSpecName "kube-api-access-9pfr8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:11:22.739949 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.739925 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-dshm" pod "935c23b1-953a-40cc-ad0c-bec2ef61afba" (UID: "935c23b1-953a-40cc-ad0c-bec2ef61afba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:22.739949 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.739938 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/935c23b1-953a-40cc-ad0c-bec2ef61afba-tls-certs" pod "935c23b1-953a-40cc-ad0c-bec2ef61afba" (UID: "935c23b1-953a-40cc-ad0c-bec2ef61afba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:11:22.840764 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.840724 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pfr8\" (UniqueName: \"kubernetes.io/projected/935c23b1-953a-40cc-ad0c-bec2ef61afba-kube-api-access-9pfr8\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:22.840764 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.840767 5029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/935c23b1-953a-40cc-ad0c-bec2ef61afba-dshm\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:22.841004 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.840782 5029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/935c23b1-953a-40cc-ad0c-bec2ef61afba-tls-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:22.920415 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.920376 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf"] Oct 30 13:11:22.931684 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.931655 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-3048f9/distinct-w3-3048f9-kserve-5c85898b56-d6xrf"] Oct 30 13:11:22.960673 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.960640 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r"] Oct 30 13:11:22.969153 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:22.969123 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r"] Oct 30 13:11:23.090883 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:23.090848 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="935c23b1-953a-40cc-ad0c-bec2ef61afba" path="/var/lib/kubelet/pods/935c23b1-953a-40cc-ad0c-bec2ef61afba/volumes" Oct 30 13:11:23.091443 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:23.091418 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9af7cbdb-5aee-4f11-9446-fceacd3fb135" path="/var/lib/kubelet/pods/9af7cbdb-5aee-4f11-9446-fceacd3fb135/volumes" Oct 30 13:11:23.455687 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:23.455600 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-3048f9/distinct-2-w3-3048f9-kserve-5954dc55-x7p7r" podUID="9af7cbdb-5aee-4f11-9446-fceacd3fb135" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded" Oct 30 13:11:27.430911 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:27.430871 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g"] Oct 30 13:11:27.431409 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:27.431202 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" podUID="2080a9e8-70bc-4c2e-8b67-7bb00077837b" containerName="maas-api" containerID="cri-o://38ddc19bab16e2d6edb67f66cc366e123fed7ee98c03c459e3ce713cdbdfa711" gracePeriod=30 Oct 30 13:11:27.637480 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:27.636974 5029 generic.go:356] "Generic (PLEG): container finished" podID="2080a9e8-70bc-4c2e-8b67-7bb00077837b" containerID="38ddc19bab16e2d6edb67f66cc366e123fed7ee98c03c459e3ce713cdbdfa711" exitCode=0 Oct 30 13:11:27.637480 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:27.637230 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" event={"ID":"2080a9e8-70bc-4c2e-8b67-7bb00077837b","Type":"ContainerDied","Data":"38ddc19bab16e2d6edb67f66cc366e123fed7ee98c03c459e3ce713cdbdfa711"} Oct 30 13:11:27.804172 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:27.802661 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" Oct 30 13:11:27.920793 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:27.920715 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2080a9e8-70bc-4c2e-8b67-7bb00077837b-kube-api-access-czv5p\" (UniqueName: \"kubernetes.io/projected/2080a9e8-70bc-4c2e-8b67-7bb00077837b-kube-api-access-czv5p\") pod \"2080a9e8-70bc-4c2e-8b67-7bb00077837b\" (UID: \"2080a9e8-70bc-4c2e-8b67-7bb00077837b\") " Oct 30 13:11:27.920793 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:27.920781 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2080a9e8-70bc-4c2e-8b67-7bb00077837b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2080a9e8-70bc-4c2e-8b67-7bb00077837b-maas-api-tls\") pod \"2080a9e8-70bc-4c2e-8b67-7bb00077837b\" (UID: \"2080a9e8-70bc-4c2e-8b67-7bb00077837b\") " Oct 30 13:11:27.921091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:27.920818 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2080a9e8-70bc-4c2e-8b67-7bb00077837b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2080a9e8-70bc-4c2e-8b67-7bb00077837b-metrics-certs\") pod \"2080a9e8-70bc-4c2e-8b67-7bb00077837b\" (UID: \"2080a9e8-70bc-4c2e-8b67-7bb00077837b\") " Oct 30 13:11:27.921091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:27.920902 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2080a9e8-70bc-4c2e-8b67-7bb00077837b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2080a9e8-70bc-4c2e-8b67-7bb00077837b-openshift-service-ca\") pod \"2080a9e8-70bc-4c2e-8b67-7bb00077837b\" (UID: \"2080a9e8-70bc-4c2e-8b67-7bb00077837b\") " Oct 30 13:11:27.922020 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:27.921977 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2080a9e8-70bc-4c2e-8b67-7bb00077837b-openshift-service-ca" pod "2080a9e8-70bc-4c2e-8b67-7bb00077837b" (UID: "2080a9e8-70bc-4c2e-8b67-7bb00077837b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:11:27.952173 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:27.944924 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2080a9e8-70bc-4c2e-8b67-7bb00077837b-maas-api-tls" pod "2080a9e8-70bc-4c2e-8b67-7bb00077837b" (UID: "2080a9e8-70bc-4c2e-8b67-7bb00077837b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:11:27.964325 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:27.955961 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2080a9e8-70bc-4c2e-8b67-7bb00077837b-metrics-certs" pod "2080a9e8-70bc-4c2e-8b67-7bb00077837b" (UID: "2080a9e8-70bc-4c2e-8b67-7bb00077837b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:11:27.964325 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:27.959440 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2080a9e8-70bc-4c2e-8b67-7bb00077837b-kube-api-access-czv5p" pod "2080a9e8-70bc-4c2e-8b67-7bb00077837b" (UID: "2080a9e8-70bc-4c2e-8b67-7bb00077837b"). InnerVolumeSpecName "kube-api-access-czv5p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:11:28.022847 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:28.022744 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czv5p\" (UniqueName: \"kubernetes.io/projected/2080a9e8-70bc-4c2e-8b67-7bb00077837b-kube-api-access-czv5p\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:28.022847 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:28.022790 5029 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2080a9e8-70bc-4c2e-8b67-7bb00077837b-maas-api-tls\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:28.022847 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:28.022805 5029 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2080a9e8-70bc-4c2e-8b67-7bb00077837b-metrics-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:28.022847 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:28.022818 5029 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2080a9e8-70bc-4c2e-8b67-7bb00077837b-openshift-service-ca\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:28.646584 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:28.646558 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" Oct 30 13:11:28.647019 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:28.646551 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g" event={"ID":"2080a9e8-70bc-4c2e-8b67-7bb00077837b","Type":"ContainerDied","Data":"207e6b973337a057da7a0b513468395e00b53d3e87a0aed97cc3bb29384392e3"} Oct 30 13:11:28.647019 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:28.646695 5029 scope.go:122] "RemoveContainer" containerID="38ddc19bab16e2d6edb67f66cc366e123fed7ee98c03c459e3ce713cdbdfa711" Oct 30 13:11:28.703339 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:28.703310 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g"] Oct 30 13:11:28.714430 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:28.714404 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-42ee9d-7bfb994b65-8hx9g"] Oct 30 13:11:29.090840 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:29.090799 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2080a9e8-70bc-4c2e-8b67-7bb00077837b" path="/var/lib/kubelet/pods/2080a9e8-70bc-4c2e-8b67-7bb00077837b/volumes" Oct 30 13:11:37.127688 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:37.127635 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp"] Oct 30 13:11:37.129779 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:37.129736 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" podUID="0e89ac39-4946-4c8a-bbc0-43c2f8487aec" containerName="istio-proxy" containerID="cri-o://f98fa61e5489a42ad058e2b36ad9cb9383b30e32dc86bb6cd84155ca5ef05585" gracePeriod=30 Oct 30 13:11:37.154856 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:37.154823 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp"] Oct 30 13:11:37.155237 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:37.155209 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" podUID="ac96e53b-9dec-4b7a-bd58-833239583b83" containerName="istio-proxy" containerID="cri-o://8f1b3e7f027ed732bc0c4f9fd317e8648356160e5f507533e9ebcceb3089197f" gracePeriod=30 Oct 30 13:11:37.160960 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:37.160936 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27"] Oct 30 13:11:37.161146 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:37.161124 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" podUID="b80d9479-2325-40f0-93fd-329c462b629c" containerName="istio-proxy" containerID="cri-o://78bde478d6fe6552fa74543291ed1528f79c109e3d1c3612476feec179826218" gracePeriod=30 Oct 30 13:11:39.986590 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:39.986552 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp"] Oct 30 13:11:39.987407 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:39.987369 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" podUID="7717496d-5ae0-421f-80c8-b89810a6b1c6" containerName="maas-api" containerID="cri-o://94794eac749679d680fc2ed8f63e4ba43abc520fb76c6913e9b6ade33525c650" gracePeriod=30 Oct 30 13:11:40.521190 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:40.521024 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" Oct 30 13:11:40.567448 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:40.565168 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7717496d-5ae0-421f-80c8-b89810a6b1c6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7717496d-5ae0-421f-80c8-b89810a6b1c6-metrics-certs\") pod \"7717496d-5ae0-421f-80c8-b89810a6b1c6\" (UID: \"7717496d-5ae0-421f-80c8-b89810a6b1c6\") " Oct 30 13:11:40.567448 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:40.565357 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7717496d-5ae0-421f-80c8-b89810a6b1c6-kube-api-access-bs5nf\" (UniqueName: \"kubernetes.io/projected/7717496d-5ae0-421f-80c8-b89810a6b1c6-kube-api-access-bs5nf\") pod \"7717496d-5ae0-421f-80c8-b89810a6b1c6\" (UID: \"7717496d-5ae0-421f-80c8-b89810a6b1c6\") " Oct 30 13:11:40.567448 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:40.565468 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7717496d-5ae0-421f-80c8-b89810a6b1c6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7717496d-5ae0-421f-80c8-b89810a6b1c6-maas-api-tls\") pod \"7717496d-5ae0-421f-80c8-b89810a6b1c6\" (UID: \"7717496d-5ae0-421f-80c8-b89810a6b1c6\") " Oct 30 13:11:40.567448 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:40.565551 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7717496d-5ae0-421f-80c8-b89810a6b1c6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7717496d-5ae0-421f-80c8-b89810a6b1c6-openshift-service-ca\") pod \"7717496d-5ae0-421f-80c8-b89810a6b1c6\" (UID: \"7717496d-5ae0-421f-80c8-b89810a6b1c6\") " Oct 30 13:11:40.568447 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:40.568420 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7717496d-5ae0-421f-80c8-b89810a6b1c6-openshift-service-ca" pod "7717496d-5ae0-421f-80c8-b89810a6b1c6" (UID: "7717496d-5ae0-421f-80c8-b89810a6b1c6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:11:40.576573 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:40.576539 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7717496d-5ae0-421f-80c8-b89810a6b1c6-metrics-certs" pod "7717496d-5ae0-421f-80c8-b89810a6b1c6" (UID: "7717496d-5ae0-421f-80c8-b89810a6b1c6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:11:40.578908 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:40.578467 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7717496d-5ae0-421f-80c8-b89810a6b1c6-kube-api-access-bs5nf" pod "7717496d-5ae0-421f-80c8-b89810a6b1c6" (UID: "7717496d-5ae0-421f-80c8-b89810a6b1c6"). InnerVolumeSpecName "kube-api-access-bs5nf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:11:40.578908 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:40.578857 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7717496d-5ae0-421f-80c8-b89810a6b1c6-maas-api-tls" pod "7717496d-5ae0-421f-80c8-b89810a6b1c6" (UID: "7717496d-5ae0-421f-80c8-b89810a6b1c6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:11:40.667459 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:40.667423 5029 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7717496d-5ae0-421f-80c8-b89810a6b1c6-maas-api-tls\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:40.667459 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:40.667457 5029 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7717496d-5ae0-421f-80c8-b89810a6b1c6-openshift-service-ca\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:40.667644 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:40.667468 5029 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7717496d-5ae0-421f-80c8-b89810a6b1c6-metrics-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:40.667644 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:40.667477 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bs5nf\" (UniqueName: \"kubernetes.io/projected/7717496d-5ae0-421f-80c8-b89810a6b1c6-kube-api-access-bs5nf\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:40.762419 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:40.746388 5029 generic.go:356] "Generic (PLEG): container finished" podID="7717496d-5ae0-421f-80c8-b89810a6b1c6" containerID="94794eac749679d680fc2ed8f63e4ba43abc520fb76c6913e9b6ade33525c650" exitCode=0 Oct 30 13:11:40.762419 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:40.746514 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" event={"ID":"7717496d-5ae0-421f-80c8-b89810a6b1c6","Type":"ContainerDied","Data":"94794eac749679d680fc2ed8f63e4ba43abc520fb76c6913e9b6ade33525c650"} Oct 30 13:11:40.762419 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:40.746622 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" event={"ID":"7717496d-5ae0-421f-80c8-b89810a6b1c6","Type":"ContainerDied","Data":"e3757ef9fa760de2cf3b995d166a49f5c56c7514f5651a3694d691c61362e8ee"} Oct 30 13:11:40.762419 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:40.746688 5029 scope.go:122] "RemoveContainer" containerID="94794eac749679d680fc2ed8f63e4ba43abc520fb76c6913e9b6ade33525c650" Oct 30 13:11:40.762419 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:40.746949 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp" Oct 30 13:11:40.782312 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:40.782280 5029 scope.go:122] "RemoveContainer" containerID="94794eac749679d680fc2ed8f63e4ba43abc520fb76c6913e9b6ade33525c650" Oct 30 13:11:40.782642 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:11:40.782620 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94794eac749679d680fc2ed8f63e4ba43abc520fb76c6913e9b6ade33525c650\": container with ID starting with 94794eac749679d680fc2ed8f63e4ba43abc520fb76c6913e9b6ade33525c650 not found: ID does not exist" containerID="94794eac749679d680fc2ed8f63e4ba43abc520fb76c6913e9b6ade33525c650" Oct 30 13:11:40.782744 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:40.782654 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94794eac749679d680fc2ed8f63e4ba43abc520fb76c6913e9b6ade33525c650"} err="failed to get container status \"94794eac749679d680fc2ed8f63e4ba43abc520fb76c6913e9b6ade33525c650\": rpc error: code = NotFound desc = could not find container \"94794eac749679d680fc2ed8f63e4ba43abc520fb76c6913e9b6ade33525c650\": container with ID starting with 94794eac749679d680fc2ed8f63e4ba43abc520fb76c6913e9b6ade33525c650 not found: ID does not exist" Oct 30 13:11:40.826245 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:40.826165 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp"] Oct 30 13:11:40.827973 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:40.827936 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-3048f9-b48bf964f-nlcnp"] Oct 30 13:11:41.092086 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:41.092036 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7717496d-5ae0-421f-80c8-b89810a6b1c6" path="/var/lib/kubelet/pods/7717496d-5ae0-421f-80c8-b89810a6b1c6/volumes" Oct 30 13:11:41.223512 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:41.223480 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/2.log" Oct 30 13:11:41.225278 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:41.225253 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/2.log" Oct 30 13:11:42.456341 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.456308 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:11:42.486672 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.486645 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:11:42.494006 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.493975 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-podinfo\") pod \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " Oct 30 13:11:42.494154 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.494096 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-kube-api-access-sm7w4\" (UniqueName: \"kubernetes.io/projected/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-kube-api-access-sm7w4\") pod \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " Oct 30 13:11:42.494154 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.494147 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-workload-certs\") pod \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " Oct 30 13:11:42.494284 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.494176 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istiod-ca-cert\") pod \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " Oct 30 13:11:42.494357 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.494336 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-workload-socket\") pod \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " Oct 30 13:11:42.494418 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.494385 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-credential-socket\") pod \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " Oct 30 13:11:42.494477 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.494458 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-workload-certs" pod "0e89ac39-4946-4c8a-bbc0-43c2f8487aec" (UID: "0e89ac39-4946-4c8a-bbc0-43c2f8487aec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:42.494535 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.494525 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-token\" (UniqueName: \"kubernetes.io/projected/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-token\") pod \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " Oct 30 13:11:42.494610 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.494592 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-data\") pod \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " Oct 30 13:11:42.494674 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.494641 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-envoy\") pod \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\" (UID: \"0e89ac39-4946-4c8a-bbc0-43c2f8487aec\") " Oct 30 13:11:42.494674 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.494654 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-workload-socket" pod "0e89ac39-4946-4c8a-bbc0-43c2f8487aec" (UID: "0e89ac39-4946-4c8a-bbc0-43c2f8487aec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:42.494774 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.494748 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istiod-ca-cert" pod "0e89ac39-4946-4c8a-bbc0-43c2f8487aec" (UID: "0e89ac39-4946-4c8a-bbc0-43c2f8487aec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:11:42.495131 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.495101 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-credential-socket" pod "0e89ac39-4946-4c8a-bbc0-43c2f8487aec" (UID: "0e89ac39-4946-4c8a-bbc0-43c2f8487aec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:42.495451 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.495430 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-workload-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.495553 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.495456 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istiod-ca-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.495553 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.495470 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-workload-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.495553 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.495486 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-credential-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.495713 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.495576 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-data" pod "0e89ac39-4946-4c8a-bbc0-43c2f8487aec" (UID: "0e89ac39-4946-4c8a-bbc0-43c2f8487aec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:42.496643 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.496614 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-podinfo" pod "0e89ac39-4946-4c8a-bbc0-43c2f8487aec" (UID: "0e89ac39-4946-4c8a-bbc0-43c2f8487aec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 13:11:42.496808 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.496784 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-kube-api-access-sm7w4" pod "0e89ac39-4946-4c8a-bbc0-43c2f8487aec" (UID: "0e89ac39-4946-4c8a-bbc0-43c2f8487aec"). InnerVolumeSpecName "kube-api-access-sm7w4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:11:42.497293 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.497265 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-token" pod "0e89ac39-4946-4c8a-bbc0-43c2f8487aec" (UID: "0e89ac39-4946-4c8a-bbc0-43c2f8487aec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:11:42.497384 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.497352 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-envoy" pod "0e89ac39-4946-4c8a-bbc0-43c2f8487aec" (UID: "0e89ac39-4946-4c8a-bbc0-43c2f8487aec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:42.509198 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.509178 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:11:42.596509 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.596479 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-istio-envoy\") pod \"b80d9479-2325-40f0-93fd-329c462b629c\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " Oct 30 13:11:42.596692 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.596525 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac96e53b-9dec-4b7a-bd58-833239583b83-kube-api-access-ftwxk\" (UniqueName: \"kubernetes.io/projected/ac96e53b-9dec-4b7a-bd58-833239583b83-kube-api-access-ftwxk\") pod \"ac96e53b-9dec-4b7a-bd58-833239583b83\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " Oct 30 13:11:42.596692 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.596573 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-workload-socket\") pod \"b80d9479-2325-40f0-93fd-329c462b629c\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " Oct 30 13:11:42.596692 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.596608 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-workload-certs\") pod \"b80d9479-2325-40f0-93fd-329c462b629c\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " Oct 30 13:11:42.596692 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.596648 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-credential-socket\") pod \"ac96e53b-9dec-4b7a-bd58-833239583b83\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " Oct 30 13:11:42.596692 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.596685 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-token\" (UniqueName: \"kubernetes.io/projected/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-token\") pod \"ac96e53b-9dec-4b7a-bd58-833239583b83\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " Oct 30 13:11:42.597016 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.596714 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac96e53b-9dec-4b7a-bd58-833239583b83-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac96e53b-9dec-4b7a-bd58-833239583b83-istiod-ca-cert\") pod \"ac96e53b-9dec-4b7a-bd58-833239583b83\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " Oct 30 13:11:42.597016 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.596744 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-envoy\") pod \"ac96e53b-9dec-4b7a-bd58-833239583b83\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " Oct 30 13:11:42.597016 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.596774 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-podinfo\") pod \"ac96e53b-9dec-4b7a-bd58-833239583b83\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " Oct 30 13:11:42.597016 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.596821 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b80d9479-2325-40f0-93fd-329c462b629c-kube-api-access-lnnn4\" (UniqueName: \"kubernetes.io/projected/b80d9479-2325-40f0-93fd-329c462b629c-kube-api-access-lnnn4\") pod \"b80d9479-2325-40f0-93fd-329c462b629c\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " Oct 30 13:11:42.597016 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.596890 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-workload-socket" pod "b80d9479-2325-40f0-93fd-329c462b629c" (UID: "b80d9479-2325-40f0-93fd-329c462b629c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:42.597016 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.596919 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-data\") pod \"ac96e53b-9dec-4b7a-bd58-833239583b83\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " Oct 30 13:11:42.597016 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.596981 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b80d9479-2325-40f0-93fd-329c462b629c-istio-token\" (UniqueName: \"kubernetes.io/projected/b80d9479-2325-40f0-93fd-329c462b629c-istio-token\") pod \"b80d9479-2325-40f0-93fd-329c462b629c\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " Oct 30 13:11:42.597381 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.597028 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b80d9479-2325-40f0-93fd-329c462b629c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b80d9479-2325-40f0-93fd-329c462b629c-istiod-ca-cert\") pod \"b80d9479-2325-40f0-93fd-329c462b629c\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " Oct 30 13:11:42.597381 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.597101 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-istio-data\") pod \"b80d9479-2325-40f0-93fd-329c462b629c\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " Oct 30 13:11:42.597381 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.597155 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-workload-certs\") pod \"ac96e53b-9dec-4b7a-bd58-833239583b83\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " Oct 30 13:11:42.597381 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.597222 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-workload-socket\") pod \"ac96e53b-9dec-4b7a-bd58-833239583b83\" (UID: \"ac96e53b-9dec-4b7a-bd58-833239583b83\") " Oct 30 13:11:42.597381 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.597328 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-credential-socket\") pod \"b80d9479-2325-40f0-93fd-329c462b629c\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " Oct 30 13:11:42.597381 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.597363 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b80d9479-2325-40f0-93fd-329c462b629c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b80d9479-2325-40f0-93fd-329c462b629c-istio-podinfo\") pod \"b80d9479-2325-40f0-93fd-329c462b629c\" (UID: \"b80d9479-2325-40f0-93fd-329c462b629c\") " Oct 30 13:11:42.598488 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.597154 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-credential-socket" pod "ac96e53b-9dec-4b7a-bd58-833239583b83" (UID: "ac96e53b-9dec-4b7a-bd58-833239583b83"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:42.598488 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.597456 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac96e53b-9dec-4b7a-bd58-833239583b83-istiod-ca-cert" pod "ac96e53b-9dec-4b7a-bd58-833239583b83" (UID: "ac96e53b-9dec-4b7a-bd58-833239583b83"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:11:42.598488 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.597941 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-istio-data" pod "b80d9479-2325-40f0-93fd-329c462b629c" (UID: "b80d9479-2325-40f0-93fd-329c462b629c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:42.598488 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.598113 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-data" pod "ac96e53b-9dec-4b7a-bd58-833239583b83" (UID: "ac96e53b-9dec-4b7a-bd58-833239583b83"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:42.598488 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.598028 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b80d9479-2325-40f0-93fd-329c462b629c-istiod-ca-cert" pod "b80d9479-2325-40f0-93fd-329c462b629c" (UID: "b80d9479-2325-40f0-93fd-329c462b629c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:11:42.598488 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.598181 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-envoy\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.598488 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.598232 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-podinfo\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.598488 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.598248 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sm7w4\" (UniqueName: \"kubernetes.io/projected/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-kube-api-access-sm7w4\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.598488 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.598264 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.598488 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.598278 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-workload-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.598488 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.598263 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-workload-certs" pod "ac96e53b-9dec-4b7a-bd58-833239583b83" (UID: "ac96e53b-9dec-4b7a-bd58-833239583b83"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:42.598488 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.598290 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e89ac39-4946-4c8a-bbc0-43c2f8487aec-istio-data\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.598488 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.598326 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-workload-socket" pod "ac96e53b-9dec-4b7a-bd58-833239583b83" (UID: "ac96e53b-9dec-4b7a-bd58-833239583b83"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:42.598488 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.598455 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-credential-socket" pod "b80d9479-2325-40f0-93fd-329c462b629c" (UID: "b80d9479-2325-40f0-93fd-329c462b629c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:42.599652 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.599597 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-workload-certs" pod "b80d9479-2325-40f0-93fd-329c462b629c" (UID: "b80d9479-2325-40f0-93fd-329c462b629c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:42.600100 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.600051 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac96e53b-9dec-4b7a-bd58-833239583b83-kube-api-access-ftwxk" pod "ac96e53b-9dec-4b7a-bd58-833239583b83" (UID: "ac96e53b-9dec-4b7a-bd58-833239583b83"). InnerVolumeSpecName "kube-api-access-ftwxk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:11:42.600325 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.600302 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-token" pod "ac96e53b-9dec-4b7a-bd58-833239583b83" (UID: "ac96e53b-9dec-4b7a-bd58-833239583b83"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:11:42.600570 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.600536 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-istio-envoy" pod "b80d9479-2325-40f0-93fd-329c462b629c" (UID: "b80d9479-2325-40f0-93fd-329c462b629c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:42.600756 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.600725 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-podinfo" pod "ac96e53b-9dec-4b7a-bd58-833239583b83" (UID: "ac96e53b-9dec-4b7a-bd58-833239583b83"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 13:11:42.600864 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.600788 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b80d9479-2325-40f0-93fd-329c462b629c-istio-podinfo" pod "b80d9479-2325-40f0-93fd-329c462b629c" (UID: "b80d9479-2325-40f0-93fd-329c462b629c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 13:11:42.601177 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.601159 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b80d9479-2325-40f0-93fd-329c462b629c-kube-api-access-lnnn4" pod "b80d9479-2325-40f0-93fd-329c462b629c" (UID: "b80d9479-2325-40f0-93fd-329c462b629c"). InnerVolumeSpecName "kube-api-access-lnnn4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:11:42.602861 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.602832 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b80d9479-2325-40f0-93fd-329c462b629c-istio-token" pod "b80d9479-2325-40f0-93fd-329c462b629c" (UID: "b80d9479-2325-40f0-93fd-329c462b629c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:11:42.602861 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.602842 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-envoy" pod "ac96e53b-9dec-4b7a-bd58-833239583b83" (UID: "ac96e53b-9dec-4b7a-bd58-833239583b83"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:42.700254 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.700169 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.700254 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.700203 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac96e53b-9dec-4b7a-bd58-833239583b83-istiod-ca-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.700254 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.700217 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-envoy\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.700254 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.700232 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-podinfo\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.700254 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.700245 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnnn4\" (UniqueName: \"kubernetes.io/projected/b80d9479-2325-40f0-93fd-329c462b629c-kube-api-access-lnnn4\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.700602 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.700259 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-istio-data\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.700602 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.700270 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b80d9479-2325-40f0-93fd-329c462b629c-istio-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.700602 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.700284 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b80d9479-2325-40f0-93fd-329c462b629c-istiod-ca-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.700602 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.700299 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-istio-data\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.700602 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.700312 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-workload-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.700602 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.700323 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-workload-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.700602 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.700332 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-credential-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.700602 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.700346 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b80d9479-2325-40f0-93fd-329c462b629c-istio-podinfo\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.700602 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.700357 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-istio-envoy\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.700602 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.700372 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftwxk\" (UniqueName: \"kubernetes.io/projected/ac96e53b-9dec-4b7a-bd58-833239583b83-kube-api-access-ftwxk\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.700602 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.700385 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b80d9479-2325-40f0-93fd-329c462b629c-workload-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.700602 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.700399 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac96e53b-9dec-4b7a-bd58-833239583b83-credential-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:42.764340 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.764300 5029 generic.go:356] "Generic (PLEG): container finished" podID="b80d9479-2325-40f0-93fd-329c462b629c" containerID="78bde478d6fe6552fa74543291ed1528f79c109e3d1c3612476feec179826218" exitCode=0 Oct 30 13:11:42.764531 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.764386 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" Oct 30 13:11:42.764531 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.764395 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" event={"ID":"b80d9479-2325-40f0-93fd-329c462b629c","Type":"ContainerDied","Data":"78bde478d6fe6552fa74543291ed1528f79c109e3d1c3612476feec179826218"} Oct 30 13:11:42.764531 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.764459 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27" event={"ID":"b80d9479-2325-40f0-93fd-329c462b629c","Type":"ContainerDied","Data":"47829c404d0bbe9a604bcbff36265cabb5caa2329cc6ba130ebf5cdccd0e2c18"} Oct 30 13:11:42.764531 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.764491 5029 scope.go:122] "RemoveContainer" containerID="78bde478d6fe6552fa74543291ed1528f79c109e3d1c3612476feec179826218" Oct 30 13:11:42.766810 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.766784 5029 generic.go:356] "Generic (PLEG): container finished" podID="ac96e53b-9dec-4b7a-bd58-833239583b83" containerID="8f1b3e7f027ed732bc0c4f9fd317e8648356160e5f507533e9ebcceb3089197f" exitCode=0 Oct 30 13:11:42.766944 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.766926 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" Oct 30 13:11:42.766944 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.766934 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" event={"ID":"ac96e53b-9dec-4b7a-bd58-833239583b83","Type":"ContainerDied","Data":"8f1b3e7f027ed732bc0c4f9fd317e8648356160e5f507533e9ebcceb3089197f"} Oct 30 13:11:42.767142 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.766983 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp" event={"ID":"ac96e53b-9dec-4b7a-bd58-833239583b83","Type":"ContainerDied","Data":"08bea501eb353f76966101ddb0dcf778f6015ed97fc949f8c2d7dc47c26321e2"} Oct 30 13:11:42.768630 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.768603 5029 generic.go:356] "Generic (PLEG): container finished" podID="0e89ac39-4946-4c8a-bbc0-43c2f8487aec" containerID="f98fa61e5489a42ad058e2b36ad9cb9383b30e32dc86bb6cd84155ca5ef05585" exitCode=0 Oct 30 13:11:42.768754 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.768678 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" event={"ID":"0e89ac39-4946-4c8a-bbc0-43c2f8487aec","Type":"ContainerDied","Data":"f98fa61e5489a42ad058e2b36ad9cb9383b30e32dc86bb6cd84155ca5ef05585"} Oct 30 13:11:42.768754 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.768713 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" event={"ID":"0e89ac39-4946-4c8a-bbc0-43c2f8487aec","Type":"ContainerDied","Data":"ece7946a2c7bd1837acf17b01bfc565300007ae21a82d405f4a2ca401b038bcc"} Oct 30 13:11:42.768870 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.768831 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp" Oct 30 13:11:42.782389 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.782370 5029 scope.go:122] "RemoveContainer" containerID="78bde478d6fe6552fa74543291ed1528f79c109e3d1c3612476feec179826218" Oct 30 13:11:42.782735 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:11:42.782693 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78bde478d6fe6552fa74543291ed1528f79c109e3d1c3612476feec179826218\": container with ID starting with 78bde478d6fe6552fa74543291ed1528f79c109e3d1c3612476feec179826218 not found: ID does not exist" containerID="78bde478d6fe6552fa74543291ed1528f79c109e3d1c3612476feec179826218" Oct 30 13:11:42.782815 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.782731 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78bde478d6fe6552fa74543291ed1528f79c109e3d1c3612476feec179826218"} err="failed to get container status \"78bde478d6fe6552fa74543291ed1528f79c109e3d1c3612476feec179826218\": rpc error: code = NotFound desc = could not find container \"78bde478d6fe6552fa74543291ed1528f79c109e3d1c3612476feec179826218\": container with ID starting with 78bde478d6fe6552fa74543291ed1528f79c109e3d1c3612476feec179826218 not found: ID does not exist" Oct 30 13:11:42.782815 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.782757 5029 scope.go:122] "RemoveContainer" containerID="8f1b3e7f027ed732bc0c4f9fd317e8648356160e5f507533e9ebcceb3089197f" Oct 30 13:11:42.792414 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.792398 5029 scope.go:122] "RemoveContainer" containerID="8f1b3e7f027ed732bc0c4f9fd317e8648356160e5f507533e9ebcceb3089197f" Oct 30 13:11:42.792686 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:11:42.792663 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f1b3e7f027ed732bc0c4f9fd317e8648356160e5f507533e9ebcceb3089197f\": container with ID starting with 8f1b3e7f027ed732bc0c4f9fd317e8648356160e5f507533e9ebcceb3089197f not found: ID does not exist" containerID="8f1b3e7f027ed732bc0c4f9fd317e8648356160e5f507533e9ebcceb3089197f" Oct 30 13:11:42.792772 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.792692 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f1b3e7f027ed732bc0c4f9fd317e8648356160e5f507533e9ebcceb3089197f"} err="failed to get container status \"8f1b3e7f027ed732bc0c4f9fd317e8648356160e5f507533e9ebcceb3089197f\": rpc error: code = NotFound desc = could not find container \"8f1b3e7f027ed732bc0c4f9fd317e8648356160e5f507533e9ebcceb3089197f\": container with ID starting with 8f1b3e7f027ed732bc0c4f9fd317e8648356160e5f507533e9ebcceb3089197f not found: ID does not exist" Oct 30 13:11:42.792772 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.792707 5029 scope.go:122] "RemoveContainer" containerID="f98fa61e5489a42ad058e2b36ad9cb9383b30e32dc86bb6cd84155ca5ef05585" Oct 30 13:11:42.801487 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.801471 5029 scope.go:122] "RemoveContainer" containerID="f98fa61e5489a42ad058e2b36ad9cb9383b30e32dc86bb6cd84155ca5ef05585" Oct 30 13:11:42.801736 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:11:42.801719 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f98fa61e5489a42ad058e2b36ad9cb9383b30e32dc86bb6cd84155ca5ef05585\": container with ID starting with f98fa61e5489a42ad058e2b36ad9cb9383b30e32dc86bb6cd84155ca5ef05585 not found: ID does not exist" containerID="f98fa61e5489a42ad058e2b36ad9cb9383b30e32dc86bb6cd84155ca5ef05585" Oct 30 13:11:42.801807 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.801746 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f98fa61e5489a42ad058e2b36ad9cb9383b30e32dc86bb6cd84155ca5ef05585"} err="failed to get container status \"f98fa61e5489a42ad058e2b36ad9cb9383b30e32dc86bb6cd84155ca5ef05585\": rpc error: code = NotFound desc = could not find container \"f98fa61e5489a42ad058e2b36ad9cb9383b30e32dc86bb6cd84155ca5ef05585\": container with ID starting with f98fa61e5489a42ad058e2b36ad9cb9383b30e32dc86bb6cd84155ca5ef05585 not found: ID does not exist" Oct 30 13:11:42.822675 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.822647 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp"] Oct 30 13:11:42.828335 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.828312 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-d8ckp"] Oct 30 13:11:42.881263 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.881233 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27"] Oct 30 13:11:42.884318 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.884291 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-jsc27"] Oct 30 13:11:42.919318 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.919287 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp"] Oct 30 13:11:42.925795 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:42.925770 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-42ee9d-openshift-default-74dd7567f9-74kbp"] Oct 30 13:11:43.090304 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:43.090262 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e89ac39-4946-4c8a-bbc0-43c2f8487aec" path="/var/lib/kubelet/pods/0e89ac39-4946-4c8a-bbc0-43c2f8487aec/volumes" Oct 30 13:11:43.090962 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:43.090935 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ac96e53b-9dec-4b7a-bd58-833239583b83" path="/var/lib/kubelet/pods/ac96e53b-9dec-4b7a-bd58-833239583b83/volumes" Oct 30 13:11:43.091533 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:43.091506 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b80d9479-2325-40f0-93fd-329c462b629c" path="/var/lib/kubelet/pods/b80d9479-2325-40f0-93fd-329c462b629c/volumes" Oct 30 13:11:44.885269 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:11:44.885225 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:11:51.009092 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:50.969789 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4"] Oct 30 13:11:51.009955 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:51.009929 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx"] Oct 30 13:11:51.010214 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:51.010179 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6"] Oct 30 13:11:51.014103 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:51.010393 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" podUID="fd2c4ae1-fbe5-46c9-82c7-f4f418994e31" containerName="istio-proxy" containerID="cri-o://ddd3d6dd30cbd234af3fe4d9dfdf59a21e4235a0371106bb139d83d03ea6e78e" gracePeriod=30 Oct 30 13:11:51.014103 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:51.010573 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" podUID="6328797e-74fc-4877-acac-0554466d541a" containerName="istio-proxy" containerID="cri-o://a963497172e5ae9baaaf5151b01a4559180508786066a5c10b6e197013912da6" gracePeriod=30 Oct 30 13:11:51.088085 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:51.080039 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5"] Oct 30 13:11:51.127092 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:51.108246 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" podUID="617de128-a9b7-428c-acec-3753566fba40" containerName="istio-proxy" containerID="cri-o://b429d7932babad8e355fb75798ddedf556f8288ad0f8743f9cc6e3754b27001f" gracePeriod=30 Oct 30 13:11:51.151778 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:51.151714 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" podUID="184f5837-5aee-46b3-856c-4c3db17a7877" containerName="istio-proxy" containerID="cri-o://3e6b43ce3f5902f8195efed31f769bd2ac7cec5d28d618c2e367e91202a0ef99" gracePeriod=30 Oct 30 13:11:51.727909 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:51.727876 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6"] Oct 30 13:11:51.736790 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:51.736763 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q"] Oct 30 13:11:56.858314 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.857962 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:11:56.912573 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.912503 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:11:56.963983 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.963951 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:11:56.981423 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.981396 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-istio-data\") pod \"617de128-a9b7-428c-acec-3753566fba40\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " Oct 30 13:11:56.981570 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.981428 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/617de128-a9b7-428c-acec-3753566fba40-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/617de128-a9b7-428c-acec-3753566fba40-istio-podinfo\") pod \"617de128-a9b7-428c-acec-3753566fba40\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " Oct 30 13:11:56.981570 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.981484 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/617de128-a9b7-428c-acec-3753566fba40-istio-token\" (UniqueName: \"kubernetes.io/projected/617de128-a9b7-428c-acec-3753566fba40-istio-token\") pod \"617de128-a9b7-428c-acec-3753566fba40\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " Oct 30 13:11:56.981690 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.981632 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-workload-socket\") pod \"617de128-a9b7-428c-acec-3753566fba40\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " Oct 30 13:11:56.981690 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.981672 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-credential-socket\") pod \"617de128-a9b7-428c-acec-3753566fba40\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " Oct 30 13:11:56.981690 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.981676 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-istio-data" pod "617de128-a9b7-428c-acec-3753566fba40" (UID: "617de128-a9b7-428c-acec-3753566fba40"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:56.981848 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.981720 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-istio-envoy\") pod \"617de128-a9b7-428c-acec-3753566fba40\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " Oct 30 13:11:56.981903 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.981854 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/617de128-a9b7-428c-acec-3753566fba40-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/617de128-a9b7-428c-acec-3753566fba40-istiod-ca-cert\") pod \"617de128-a9b7-428c-acec-3753566fba40\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " Oct 30 13:11:56.981960 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.981903 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-workload-certs\") pod \"617de128-a9b7-428c-acec-3753566fba40\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " Oct 30 13:11:56.982012 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.981993 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-credential-socket" pod "617de128-a9b7-428c-acec-3753566fba40" (UID: "617de128-a9b7-428c-acec-3753566fba40"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:56.982012 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.981999 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/617de128-a9b7-428c-acec-3753566fba40-kube-api-access-hvg2n\" (UniqueName: \"kubernetes.io/projected/617de128-a9b7-428c-acec-3753566fba40-kube-api-access-hvg2n\") pod \"617de128-a9b7-428c-acec-3753566fba40\" (UID: \"617de128-a9b7-428c-acec-3753566fba40\") " Oct 30 13:11:56.982245 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.982197 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-workload-certs" pod "617de128-a9b7-428c-acec-3753566fba40" (UID: "617de128-a9b7-428c-acec-3753566fba40"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:56.982457 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.982382 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/617de128-a9b7-428c-acec-3753566fba40-istiod-ca-cert" pod "617de128-a9b7-428c-acec-3753566fba40" (UID: "617de128-a9b7-428c-acec-3753566fba40"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:11:56.982457 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.982411 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-workload-socket" pod "617de128-a9b7-428c-acec-3753566fba40" (UID: "617de128-a9b7-428c-acec-3753566fba40"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:56.982656 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.982645 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-istio-data\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:56.982719 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.982666 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-workload-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:56.982719 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.982683 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-credential-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:56.982719 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.982697 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/617de128-a9b7-428c-acec-3753566fba40-istiod-ca-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:56.982719 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.982709 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-workload-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:56.984261 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.984236 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/617de128-a9b7-428c-acec-3753566fba40-istio-token" pod "617de128-a9b7-428c-acec-3753566fba40" (UID: "617de128-a9b7-428c-acec-3753566fba40"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:11:56.984382 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.984359 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/617de128-a9b7-428c-acec-3753566fba40-istio-podinfo" pod "617de128-a9b7-428c-acec-3753566fba40" (UID: "617de128-a9b7-428c-acec-3753566fba40"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 13:11:56.984533 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.984516 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-istio-envoy" pod "617de128-a9b7-428c-acec-3753566fba40" (UID: "617de128-a9b7-428c-acec-3753566fba40"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:56.984616 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:56.984601 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/617de128-a9b7-428c-acec-3753566fba40-kube-api-access-hvg2n" pod "617de128-a9b7-428c-acec-3753566fba40" (UID: "617de128-a9b7-428c-acec-3753566fba40"). InnerVolumeSpecName "kube-api-access-hvg2n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:11:57.019465 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.019431 5029 generic.go:356] "Generic (PLEG): container finished" podID="fd2c4ae1-fbe5-46c9-82c7-f4f418994e31" containerID="ddd3d6dd30cbd234af3fe4d9dfdf59a21e4235a0371106bb139d83d03ea6e78e" exitCode=0 Oct 30 13:11:57.019623 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.019484 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" event={"ID":"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31","Type":"ContainerDied","Data":"ddd3d6dd30cbd234af3fe4d9dfdf59a21e4235a0371106bb139d83d03ea6e78e"} Oct 30 13:11:57.022488 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.022462 5029 generic.go:356] "Generic (PLEG): container finished" podID="617de128-a9b7-428c-acec-3753566fba40" containerID="b429d7932babad8e355fb75798ddedf556f8288ad0f8743f9cc6e3754b27001f" exitCode=0 Oct 30 13:11:57.022888 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.022518 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" Oct 30 13:11:57.022988 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.022520 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" event={"ID":"617de128-a9b7-428c-acec-3753566fba40","Type":"ContainerDied","Data":"b429d7932babad8e355fb75798ddedf556f8288ad0f8743f9cc6e3754b27001f"} Oct 30 13:11:57.023054 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.022987 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5" event={"ID":"617de128-a9b7-428c-acec-3753566fba40","Type":"ContainerDied","Data":"b39c196422216915701699e1bb2c12f258b2c5eb82958732e63b21ec2abf469c"} Oct 30 13:11:57.023054 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.023029 5029 scope.go:122] "RemoveContainer" containerID="b429d7932babad8e355fb75798ddedf556f8288ad0f8743f9cc6e3754b27001f" Oct 30 13:11:57.024809 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.024791 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:11:57.025729 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.025036 5029 generic.go:356] "Generic (PLEG): container finished" podID="184f5837-5aee-46b3-856c-4c3db17a7877" containerID="3e6b43ce3f5902f8195efed31f769bd2ac7cec5d28d618c2e367e91202a0ef99" exitCode=0 Oct 30 13:11:57.025729 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.025113 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" Oct 30 13:11:57.025729 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.025188 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" event={"ID":"184f5837-5aee-46b3-856c-4c3db17a7877","Type":"ContainerDied","Data":"3e6b43ce3f5902f8195efed31f769bd2ac7cec5d28d618c2e367e91202a0ef99"} Oct 30 13:11:57.025729 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.025230 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4" event={"ID":"184f5837-5aee-46b3-856c-4c3db17a7877","Type":"ContainerDied","Data":"f025b6076a887eab0c15c8493833a884902c5f8b7c29cfcae61481ea125b591c"} Oct 30 13:11:57.027007 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.026982 5029 generic.go:356] "Generic (PLEG): container finished" podID="6328797e-74fc-4877-acac-0554466d541a" containerID="a963497172e5ae9baaaf5151b01a4559180508786066a5c10b6e197013912da6" exitCode=0 Oct 30 13:11:57.027127 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.027053 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" event={"ID":"6328797e-74fc-4877-acac-0554466d541a","Type":"ContainerDied","Data":"a963497172e5ae9baaaf5151b01a4559180508786066a5c10b6e197013912da6"} Oct 30 13:11:57.027127 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.027122 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" event={"ID":"6328797e-74fc-4877-acac-0554466d541a","Type":"ContainerDied","Data":"cbeed0e47cb3cddf9a65b3b7bbf321c8d8f77a1b4391e3ed4699924eddff8205"} Oct 30 13:11:57.027252 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.027191 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6" Oct 30 13:11:57.033692 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.033669 5029 scope.go:122] "RemoveContainer" containerID="b429d7932babad8e355fb75798ddedf556f8288ad0f8743f9cc6e3754b27001f" Oct 30 13:11:57.034140 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:11:57.034118 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b429d7932babad8e355fb75798ddedf556f8288ad0f8743f9cc6e3754b27001f\": container with ID starting with b429d7932babad8e355fb75798ddedf556f8288ad0f8743f9cc6e3754b27001f not found: ID does not exist" containerID="b429d7932babad8e355fb75798ddedf556f8288ad0f8743f9cc6e3754b27001f" Oct 30 13:11:57.034298 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.034266 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b429d7932babad8e355fb75798ddedf556f8288ad0f8743f9cc6e3754b27001f"} err="failed to get container status \"b429d7932babad8e355fb75798ddedf556f8288ad0f8743f9cc6e3754b27001f\": rpc error: code = NotFound desc = could not find container \"b429d7932babad8e355fb75798ddedf556f8288ad0f8743f9cc6e3754b27001f\": container with ID starting with b429d7932babad8e355fb75798ddedf556f8288ad0f8743f9cc6e3754b27001f not found: ID does not exist" Oct 30 13:11:57.034408 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.034396 5029 scope.go:122] "RemoveContainer" containerID="3e6b43ce3f5902f8195efed31f769bd2ac7cec5d28d618c2e367e91202a0ef99" Oct 30 13:11:57.085294 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.085267 5029 scope.go:122] "RemoveContainer" containerID="3e6b43ce3f5902f8195efed31f769bd2ac7cec5d28d618c2e367e91202a0ef99" Oct 30 13:11:57.086632 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:11:57.086602 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e6b43ce3f5902f8195efed31f769bd2ac7cec5d28d618c2e367e91202a0ef99\": container with ID starting with 3e6b43ce3f5902f8195efed31f769bd2ac7cec5d28d618c2e367e91202a0ef99 not found: ID does not exist" containerID="3e6b43ce3f5902f8195efed31f769bd2ac7cec5d28d618c2e367e91202a0ef99" Oct 30 13:11:57.086764 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.086645 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e6b43ce3f5902f8195efed31f769bd2ac7cec5d28d618c2e367e91202a0ef99"} err="failed to get container status \"3e6b43ce3f5902f8195efed31f769bd2ac7cec5d28d618c2e367e91202a0ef99\": rpc error: code = NotFound desc = could not find container \"3e6b43ce3f5902f8195efed31f769bd2ac7cec5d28d618c2e367e91202a0ef99\": container with ID starting with 3e6b43ce3f5902f8195efed31f769bd2ac7cec5d28d618c2e367e91202a0ef99 not found: ID does not exist" Oct 30 13:11:57.086764 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.086668 5029 scope.go:122] "RemoveContainer" containerID="a963497172e5ae9baaaf5151b01a4559180508786066a5c10b6e197013912da6" Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.096208 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-istio-data\") pod \"184f5837-5aee-46b3-856c-4c3db17a7877\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.096268 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-credential-socket\") pod \"184f5837-5aee-46b3-856c-4c3db17a7877\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.096331 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-istio-data\") pod \"6328797e-74fc-4877-acac-0554466d541a\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.096372 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6328797e-74fc-4877-acac-0554466d541a-istio-token\" (UniqueName: \"kubernetes.io/projected/6328797e-74fc-4877-acac-0554466d541a-istio-token\") pod \"6328797e-74fc-4877-acac-0554466d541a\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.096444 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6328797e-74fc-4877-acac-0554466d541a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6328797e-74fc-4877-acac-0554466d541a-istio-podinfo\") pod \"6328797e-74fc-4877-acac-0554466d541a\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.096516 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/184f5837-5aee-46b3-856c-4c3db17a7877-kube-api-access-dghcb\" (UniqueName: \"kubernetes.io/projected/184f5837-5aee-46b3-856c-4c3db17a7877-kube-api-access-dghcb\") pod \"184f5837-5aee-46b3-856c-4c3db17a7877\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.096600 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-workload-certs\") pod \"6328797e-74fc-4877-acac-0554466d541a\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.096636 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-workload-socket\") pod \"6328797e-74fc-4877-acac-0554466d541a\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.096670 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-workload-socket\") pod \"184f5837-5aee-46b3-856c-4c3db17a7877\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.096712 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6328797e-74fc-4877-acac-0554466d541a-kube-api-access-2blz7\" (UniqueName: \"kubernetes.io/projected/6328797e-74fc-4877-acac-0554466d541a-kube-api-access-2blz7\") pod \"6328797e-74fc-4877-acac-0554466d541a\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.096748 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/184f5837-5aee-46b3-856c-4c3db17a7877-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/184f5837-5aee-46b3-856c-4c3db17a7877-istio-podinfo\") pod \"184f5837-5aee-46b3-856c-4c3db17a7877\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.096815 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-workload-certs\") pod \"184f5837-5aee-46b3-856c-4c3db17a7877\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.096841 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-credential-socket" pod "184f5837-5aee-46b3-856c-4c3db17a7877" (UID: "184f5837-5aee-46b3-856c-4c3db17a7877"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.096864 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6328797e-74fc-4877-acac-0554466d541a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6328797e-74fc-4877-acac-0554466d541a-istiod-ca-cert\") pod \"6328797e-74fc-4877-acac-0554466d541a\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.096899 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-istio-envoy\") pod \"6328797e-74fc-4877-acac-0554466d541a\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.096925 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-istio-envoy\") pod \"184f5837-5aee-46b3-856c-4c3db17a7877\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.096986 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/184f5837-5aee-46b3-856c-4c3db17a7877-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/184f5837-5aee-46b3-856c-4c3db17a7877-istiod-ca-cert\") pod \"184f5837-5aee-46b3-856c-4c3db17a7877\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.097043 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-workload-socket" pod "184f5837-5aee-46b3-856c-4c3db17a7877" (UID: "184f5837-5aee-46b3-856c-4c3db17a7877"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.097198 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-data\") pod \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.097245 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-credential-socket\") pod \"6328797e-74fc-4877-acac-0554466d541a\" (UID: \"6328797e-74fc-4877-acac-0554466d541a\") " Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.097255 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-workload-certs" pod "6328797e-74fc-4877-acac-0554466d541a" (UID: "6328797e-74fc-4877-acac-0554466d541a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.097357 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-podinfo\") pod \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.097419 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/184f5837-5aee-46b3-856c-4c3db17a7877-istio-token\" (UniqueName: \"kubernetes.io/projected/184f5837-5aee-46b3-856c-4c3db17a7877-istio-token\") pod \"184f5837-5aee-46b3-856c-4c3db17a7877\" (UID: \"184f5837-5aee-46b3-856c-4c3db17a7877\") " Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.097458 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-credential-socket" pod "6328797e-74fc-4877-acac-0554466d541a" (UID: "6328797e-74fc-4877-acac-0554466d541a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.097633 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-data" pod "fd2c4ae1-fbe5-46c9-82c7-f4f418994e31" (UID: "fd2c4ae1-fbe5-46c9-82c7-f4f418994e31"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.098324 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/617de128-a9b7-428c-acec-3753566fba40-istio-envoy\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.098353 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-data\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.098367 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-credential-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.098381 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-credential-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.098395 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvg2n\" (UniqueName: \"kubernetes.io/projected/617de128-a9b7-428c-acec-3753566fba40-kube-api-access-hvg2n\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.098413 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/617de128-a9b7-428c-acec-3753566fba40-istio-podinfo\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.098429 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/617de128-a9b7-428c-acec-3753566fba40-istio-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.098441 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-workload-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.098453 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-workload-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.098970 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-workload-socket" pod "6328797e-74fc-4877-acac-0554466d541a" (UID: "6328797e-74fc-4877-acac-0554466d541a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.099511 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-istio-data" pod "6328797e-74fc-4877-acac-0554466d541a" (UID: "6328797e-74fc-4877-acac-0554466d541a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.099981 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-workload-certs" pod "184f5837-5aee-46b3-856c-4c3db17a7877" (UID: "184f5837-5aee-46b3-856c-4c3db17a7877"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.100383 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/184f5837-5aee-46b3-856c-4c3db17a7877-istiod-ca-cert" pod "184f5837-5aee-46b3-856c-4c3db17a7877" (UID: "184f5837-5aee-46b3-856c-4c3db17a7877"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.100748 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6328797e-74fc-4877-acac-0554466d541a-istiod-ca-cert" pod "6328797e-74fc-4877-acac-0554466d541a" (UID: "6328797e-74fc-4877-acac-0554466d541a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:11:57.109110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.100814 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-istio-data" pod "184f5837-5aee-46b3-856c-4c3db17a7877" (UID: "184f5837-5aee-46b3-856c-4c3db17a7877"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:57.113748 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.113510 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-podinfo" pod "fd2c4ae1-fbe5-46c9-82c7-f4f418994e31" (UID: "fd2c4ae1-fbe5-46c9-82c7-f4f418994e31"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 13:11:57.113943 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.113593 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/184f5837-5aee-46b3-856c-4c3db17a7877-kube-api-access-dghcb" pod "184f5837-5aee-46b3-856c-4c3db17a7877" (UID: "184f5837-5aee-46b3-856c-4c3db17a7877"). InnerVolumeSpecName "kube-api-access-dghcb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:11:57.113943 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.113850 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/184f5837-5aee-46b3-856c-4c3db17a7877-istio-podinfo" pod "184f5837-5aee-46b3-856c-4c3db17a7877" (UID: "184f5837-5aee-46b3-856c-4c3db17a7877"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 13:11:57.113943 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.113864 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/184f5837-5aee-46b3-856c-4c3db17a7877-istio-token" pod "184f5837-5aee-46b3-856c-4c3db17a7877" (UID: "184f5837-5aee-46b3-856c-4c3db17a7877"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:11:57.113943 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.113873 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6328797e-74fc-4877-acac-0554466d541a-istio-token" pod "6328797e-74fc-4877-acac-0554466d541a" (UID: "6328797e-74fc-4877-acac-0554466d541a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:11:57.113943 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.113885 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-istio-envoy" pod "184f5837-5aee-46b3-856c-4c3db17a7877" (UID: "184f5837-5aee-46b3-856c-4c3db17a7877"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:57.113943 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.113911 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6328797e-74fc-4877-acac-0554466d541a-kube-api-access-2blz7" pod "6328797e-74fc-4877-acac-0554466d541a" (UID: "6328797e-74fc-4877-acac-0554466d541a"). InnerVolumeSpecName "kube-api-access-2blz7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:11:57.113943 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.113941 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-istio-envoy" pod "6328797e-74fc-4877-acac-0554466d541a" (UID: "6328797e-74fc-4877-acac-0554466d541a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:57.117261 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.117240 5029 scope.go:122] "RemoveContainer" containerID="a963497172e5ae9baaaf5151b01a4559180508786066a5c10b6e197013912da6" Oct 30 13:11:57.120224 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.119373 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6328797e-74fc-4877-acac-0554466d541a-istio-podinfo" pod "6328797e-74fc-4877-acac-0554466d541a" (UID: "6328797e-74fc-4877-acac-0554466d541a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 13:11:57.122741 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:11:57.122607 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a963497172e5ae9baaaf5151b01a4559180508786066a5c10b6e197013912da6\": container with ID starting with a963497172e5ae9baaaf5151b01a4559180508786066a5c10b6e197013912da6 not found: ID does not exist" containerID="a963497172e5ae9baaaf5151b01a4559180508786066a5c10b6e197013912da6" Oct 30 13:11:57.122741 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.122636 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a963497172e5ae9baaaf5151b01a4559180508786066a5c10b6e197013912da6"} err="failed to get container status \"a963497172e5ae9baaaf5151b01a4559180508786066a5c10b6e197013912da6\": rpc error: code = NotFound desc = could not find container \"a963497172e5ae9baaaf5151b01a4559180508786066a5c10b6e197013912da6\": container with ID starting with a963497172e5ae9baaaf5151b01a4559180508786066a5c10b6e197013912da6 not found: ID does not exist" Oct 30 13:11:57.188442 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.188391 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5"] Oct 30 13:11:57.194665 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.194642 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-984s5"] Oct 30 13:11:57.199335 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.199307 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-workload-certs\") pod \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " Oct 30 13:11:57.199451 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.199359 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-token\" (UniqueName: \"kubernetes.io/projected/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-token\") pod \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " Oct 30 13:11:57.199451 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.199425 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istiod-ca-cert\") pod \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " Oct 30 13:11:57.199451 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.199445 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-kube-api-access-kvh46\" (UniqueName: \"kubernetes.io/projected/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-kube-api-access-kvh46\") pod \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " Oct 30 13:11:57.199579 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.199474 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-credential-socket\") pod \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " Oct 30 13:11:57.199579 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.199508 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-workload-socket\") pod \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " Oct 30 13:11:57.199579 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.199567 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-envoy\") pod \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\" (UID: \"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31\") " Oct 30 13:11:57.199674 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.199614 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-workload-certs" pod "fd2c4ae1-fbe5-46c9-82c7-f4f418994e31" (UID: "fd2c4ae1-fbe5-46c9-82c7-f4f418994e31"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:57.200001 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.199795 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-credential-socket" pod "fd2c4ae1-fbe5-46c9-82c7-f4f418994e31" (UID: "fd2c4ae1-fbe5-46c9-82c7-f4f418994e31"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:57.200001 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.199844 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istiod-ca-cert" pod "fd2c4ae1-fbe5-46c9-82c7-f4f418994e31" (UID: "fd2c4ae1-fbe5-46c9-82c7-f4f418994e31"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:11:57.200001 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.199947 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-workload-socket" pod "fd2c4ae1-fbe5-46c9-82c7-f4f418994e31" (UID: "fd2c4ae1-fbe5-46c9-82c7-f4f418994e31"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:57.200297 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.200196 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dghcb\" (UniqueName: \"kubernetes.io/projected/184f5837-5aee-46b3-856c-4c3db17a7877-kube-api-access-dghcb\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.200297 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.200232 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istiod-ca-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.200297 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.200247 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-credential-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.200297 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.200260 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-workload-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.200520 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.200330 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2blz7\" (UniqueName: \"kubernetes.io/projected/6328797e-74fc-4877-acac-0554466d541a-kube-api-access-2blz7\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.200520 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.200344 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-workload-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.200520 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.200357 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/184f5837-5aee-46b3-856c-4c3db17a7877-istio-podinfo\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.200520 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.200371 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-workload-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.200520 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.200384 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6328797e-74fc-4877-acac-0554466d541a-istiod-ca-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.200520 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.200397 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-istio-envoy\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.200520 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.200424 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-istio-envoy\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.200520 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.200439 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/184f5837-5aee-46b3-856c-4c3db17a7877-istiod-ca-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.200520 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.200453 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-podinfo\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.200520 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.200465 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/184f5837-5aee-46b3-856c-4c3db17a7877-istio-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.200520 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.200479 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/184f5837-5aee-46b3-856c-4c3db17a7877-istio-data\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.200520 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.200491 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-workload-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.200520 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.200516 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6328797e-74fc-4877-acac-0554466d541a-istio-data\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.201206 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.200551 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6328797e-74fc-4877-acac-0554466d541a-istio-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.201206 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.200584 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6328797e-74fc-4877-acac-0554466d541a-istio-podinfo\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.202126 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.202099 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-kube-api-access-kvh46" pod "fd2c4ae1-fbe5-46c9-82c7-f4f418994e31" (UID: "fd2c4ae1-fbe5-46c9-82c7-f4f418994e31"). InnerVolumeSpecName "kube-api-access-kvh46". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:11:57.202233 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.202137 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-token" pod "fd2c4ae1-fbe5-46c9-82c7-f4f418994e31" (UID: "fd2c4ae1-fbe5-46c9-82c7-f4f418994e31"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:11:57.202852 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.202831 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-envoy" pod "fd2c4ae1-fbe5-46c9-82c7-f4f418994e31" (UID: "fd2c4ae1-fbe5-46c9-82c7-f4f418994e31"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:11:57.301802 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.301753 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-envoy\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.301802 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.301799 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-istio-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.302029 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.301815 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvh46\" (UniqueName: \"kubernetes.io/projected/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31-kube-api-access-kvh46\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:11:57.369779 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.369694 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4"] Oct 30 13:11:57.379631 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.378497 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-fxjz4"] Oct 30 13:11:57.416090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.413437 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6"] Oct 30 13:11:57.423258 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:57.423198 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-7hlw6"] Oct 30 13:11:58.043684 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:58.043646 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" Oct 30 13:11:58.044241 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:58.043649 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx" event={"ID":"fd2c4ae1-fbe5-46c9-82c7-f4f418994e31","Type":"ContainerDied","Data":"8a68fa25957d03055387234a9d091045500e33afa74006a3d0549776dd2b573f"} Oct 30 13:11:58.044241 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:58.043783 5029 scope.go:122] "RemoveContainer" containerID="ddd3d6dd30cbd234af3fe4d9dfdf59a21e4235a0371106bb139d83d03ea6e78e" Oct 30 13:11:58.101504 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:58.101471 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx"] Oct 30 13:11:58.114476 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:58.114445 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-3048f9-openshift-default-89cb9455f-8x9hx"] Oct 30 13:11:59.089712 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:59.089682 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="184f5837-5aee-46b3-856c-4c3db17a7877" path="/var/lib/kubelet/pods/184f5837-5aee-46b3-856c-4c3db17a7877/volumes" Oct 30 13:11:59.090261 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:59.090244 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="617de128-a9b7-428c-acec-3753566fba40" path="/var/lib/kubelet/pods/617de128-a9b7-428c-acec-3753566fba40/volumes" Oct 30 13:11:59.090669 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:59.090656 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6328797e-74fc-4877-acac-0554466d541a" path="/var/lib/kubelet/pods/6328797e-74fc-4877-acac-0554466d541a/volumes" Oct 30 13:11:59.091426 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:11:59.091405 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fd2c4ae1-fbe5-46c9-82c7-f4f418994e31" path="/var/lib/kubelet/pods/fd2c4ae1-fbe5-46c9-82c7-f4f418994e31/volumes" Oct 30 13:12:06.750912 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:06.750855 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" podUID="b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" containerName="main" containerID="cri-o://b8c9908df3066256bfac36e26023aab46d96c34937e80a47b502696eb3805322" gracePeriod=45 Oct 30 13:12:06.759484 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:06.759439 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" podUID="6962e138-faa9-438c-9138-1133bdce1011" containerName="main" containerID="cri-o://42967a7990a5cd4d4d12e4ce63cf7e8cb0076366d33efef8944759e0e8463c9b" gracePeriod=45 Oct 30 13:12:06.877718 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:06.877687 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" podUID="6962e138-faa9-438c-9138-1133bdce1011" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": dial tcp 10.132.0.56:8000: connect: connection refused" Oct 30 13:12:07.055408 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.055378 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:12:07.058621 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.058599 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:12:07.112637 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.112605 5029 generic.go:356] "Generic (PLEG): container finished" podID="6962e138-faa9-438c-9138-1133bdce1011" containerID="42967a7990a5cd4d4d12e4ce63cf7e8cb0076366d33efef8944759e0e8463c9b" exitCode=0 Oct 30 13:12:07.112807 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.112685 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" event={"ID":"6962e138-faa9-438c-9138-1133bdce1011","Type":"ContainerDied","Data":"42967a7990a5cd4d4d12e4ce63cf7e8cb0076366d33efef8944759e0e8463c9b"} Oct 30 13:12:07.112807 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.112732 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" event={"ID":"6962e138-faa9-438c-9138-1133bdce1011","Type":"ContainerDied","Data":"32b492900f25285d660f326e2444ed5caabf024a506f89bfb2dd84f9414ae69c"} Oct 30 13:12:07.112807 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.112756 5029 scope.go:122] "RemoveContainer" containerID="42967a7990a5cd4d4d12e4ce63cf7e8cb0076366d33efef8944759e0e8463c9b" Oct 30 13:12:07.113404 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.113383 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q" Oct 30 13:12:07.114762 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.114649 5029 generic.go:356] "Generic (PLEG): container finished" podID="b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" containerID="b8c9908df3066256bfac36e26023aab46d96c34937e80a47b502696eb3805322" exitCode=0 Oct 30 13:12:07.114762 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.114704 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" Oct 30 13:12:07.114921 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.114766 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" event={"ID":"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2","Type":"ContainerDied","Data":"b8c9908df3066256bfac36e26023aab46d96c34937e80a47b502696eb3805322"} Oct 30 13:12:07.114921 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.114807 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6" event={"ID":"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2","Type":"ContainerDied","Data":"fdd32517a555bf8827549b4de8cf553a4a29bbf1a50b117fd907df1d00a55c89"} Oct 30 13:12:07.121036 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.121002 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6962e138-faa9-438c-9138-1133bdce1011-tls-certs\" (UniqueName: \"kubernetes.io/secret/6962e138-faa9-438c-9138-1133bdce1011-tls-certs\") pod \"6962e138-faa9-438c-9138-1133bdce1011\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " Oct 30 13:12:07.121198 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.121085 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-home\" (UniqueName: \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-home\") pod \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " Oct 30 13:12:07.121198 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.121189 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-kube-api-access-vjmhq\" (UniqueName: \"kubernetes.io/projected/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-kube-api-access-vjmhq\") pod \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " Oct 30 13:12:07.121329 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.121237 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-tls-certs\" (UniqueName: \"kubernetes.io/secret/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-tls-certs\") pod \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " Oct 30 13:12:07.121329 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.121293 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-dshm\") pod \"6962e138-faa9-438c-9138-1133bdce1011\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " Oct 30 13:12:07.121434 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.121326 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6962e138-faa9-438c-9138-1133bdce1011-kube-api-access-x4hlt\" (UniqueName: \"kubernetes.io/projected/6962e138-faa9-438c-9138-1133bdce1011-kube-api-access-x4hlt\") pod \"6962e138-faa9-438c-9138-1133bdce1011\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " Oct 30 13:12:07.121434 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.121363 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-model-cache\") pod \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " Oct 30 13:12:07.121434 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.121399 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-tmp-dir\") pod \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " Oct 30 13:12:07.121598 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.121432 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-tmp-dir\") pod \"6962e138-faa9-438c-9138-1133bdce1011\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " Oct 30 13:12:07.121598 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.121475 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-model-cache\") pod \"6962e138-faa9-438c-9138-1133bdce1011\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " Oct 30 13:12:07.121598 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.121566 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-dshm\") pod \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\" (UID: \"b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2\") " Oct 30 13:12:07.121764 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.121640 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-home\" (UniqueName: \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-home\") pod \"6962e138-faa9-438c-9138-1133bdce1011\" (UID: \"6962e138-faa9-438c-9138-1133bdce1011\") " Oct 30 13:12:07.124887 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.123484 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-home" pod "6962e138-faa9-438c-9138-1133bdce1011" (UID: "6962e138-faa9-438c-9138-1133bdce1011"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:07.124887 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.123760 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-model-cache" pod "b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" (UID: "b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:07.124887 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.124422 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-tmp-dir" pod "6962e138-faa9-438c-9138-1133bdce1011" (UID: "6962e138-faa9-438c-9138-1133bdce1011"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:07.124887 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.124531 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6962e138-faa9-438c-9138-1133bdce1011-kube-api-access-x4hlt" pod "6962e138-faa9-438c-9138-1133bdce1011" (UID: "6962e138-faa9-438c-9138-1133bdce1011"). InnerVolumeSpecName "kube-api-access-x4hlt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:12:07.125209 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.125183 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-model-cache" pod "6962e138-faa9-438c-9138-1133bdce1011" (UID: "6962e138-faa9-438c-9138-1133bdce1011"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:07.125331 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.125300 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-home" pod "b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" (UID: "b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:07.125498 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.125476 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-tmp-dir" pod "b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" (UID: "b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:07.126040 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.126017 5029 scope.go:122] "RemoveContainer" containerID="42967a7990a5cd4d4d12e4ce63cf7e8cb0076366d33efef8944759e0e8463c9b" Oct 30 13:12:07.126420 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:12:07.126391 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42967a7990a5cd4d4d12e4ce63cf7e8cb0076366d33efef8944759e0e8463c9b\": container with ID starting with 42967a7990a5cd4d4d12e4ce63cf7e8cb0076366d33efef8944759e0e8463c9b not found: ID does not exist" containerID="42967a7990a5cd4d4d12e4ce63cf7e8cb0076366d33efef8944759e0e8463c9b" Oct 30 13:12:07.126572 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.126433 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42967a7990a5cd4d4d12e4ce63cf7e8cb0076366d33efef8944759e0e8463c9b"} err="failed to get container status \"42967a7990a5cd4d4d12e4ce63cf7e8cb0076366d33efef8944759e0e8463c9b\": rpc error: code = NotFound desc = could not find container \"42967a7990a5cd4d4d12e4ce63cf7e8cb0076366d33efef8944759e0e8463c9b\": container with ID starting with 42967a7990a5cd4d4d12e4ce63cf7e8cb0076366d33efef8944759e0e8463c9b not found: ID does not exist" Oct 30 13:12:07.126572 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.126459 5029 scope.go:122] "RemoveContainer" containerID="b8c9908df3066256bfac36e26023aab46d96c34937e80a47b502696eb3805322" Oct 30 13:12:07.126848 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.126821 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-tls-certs" pod "b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" (UID: "b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:12:07.131425 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.131405 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-kube-api-access-vjmhq" pod "b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" (UID: "b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2"). InnerVolumeSpecName "kube-api-access-vjmhq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:12:07.131536 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.131518 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-dshm" pod "b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" (UID: "b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:07.131633 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.131614 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6962e138-faa9-438c-9138-1133bdce1011-tls-certs" pod "6962e138-faa9-438c-9138-1133bdce1011" (UID: "6962e138-faa9-438c-9138-1133bdce1011"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:12:07.134209 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.134172 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-dshm" pod "6962e138-faa9-438c-9138-1133bdce1011" (UID: "6962e138-faa9-438c-9138-1133bdce1011"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:07.141649 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.141619 5029 scope.go:122] "RemoveContainer" containerID="b8c9908df3066256bfac36e26023aab46d96c34937e80a47b502696eb3805322" Oct 30 13:12:07.142805 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:12:07.142773 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8c9908df3066256bfac36e26023aab46d96c34937e80a47b502696eb3805322\": container with ID starting with b8c9908df3066256bfac36e26023aab46d96c34937e80a47b502696eb3805322 not found: ID does not exist" containerID="b8c9908df3066256bfac36e26023aab46d96c34937e80a47b502696eb3805322" Oct 30 13:12:07.142911 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.142809 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8c9908df3066256bfac36e26023aab46d96c34937e80a47b502696eb3805322"} err="failed to get container status \"b8c9908df3066256bfac36e26023aab46d96c34937e80a47b502696eb3805322\": rpc error: code = NotFound desc = could not find container \"b8c9908df3066256bfac36e26023aab46d96c34937e80a47b502696eb3805322\": container with ID starting with b8c9908df3066256bfac36e26023aab46d96c34937e80a47b502696eb3805322 not found: ID does not exist" Oct 30 13:12:07.223538 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.223500 5029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6962e138-faa9-438c-9138-1133bdce1011-tls-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:07.223538 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.223532 5029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-home\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:07.223755 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.223545 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjmhq\" (UniqueName: \"kubernetes.io/projected/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-kube-api-access-vjmhq\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:07.223755 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.223560 5029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-tls-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:07.223755 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.223571 5029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-dshm\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:07.223755 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.223582 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4hlt\" (UniqueName: \"kubernetes.io/projected/6962e138-faa9-438c-9138-1133bdce1011-kube-api-access-x4hlt\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:07.223755 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.223593 5029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-model-cache\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:07.223755 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.223606 5029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-tmp-dir\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:07.223755 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.223616 5029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-tmp-dir\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:07.223755 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.223627 5029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-model-cache\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:07.223755 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.223637 5029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2-dshm\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:07.223755 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.223648 5029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6962e138-faa9-438c-9138-1133bdce1011-home\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:07.463511 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.463478 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6"] Oct 30 13:12:07.470891 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.470869 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-484607/distinct-w6-484607-kserve-7585c5f98d-8zzv6"] Oct 30 13:12:07.505808 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.505781 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q"] Oct 30 13:12:07.511631 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:07.511604 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-484607/premium-w6-484607-kserve-7f8d6b7d8f-fl54q"] Oct 30 13:12:09.089306 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:09.089267 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6962e138-faa9-438c-9138-1133bdce1011" path="/var/lib/kubelet/pods/6962e138-faa9-438c-9138-1133bdce1011/volumes" Oct 30 13:12:09.089763 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:09.089746 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" path="/var/lib/kubelet/pods/b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2/volumes" Oct 30 13:12:12.878431 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:12.878397 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x"] Oct 30 13:12:12.878922 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:12.878630 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" podUID="5f73d872-eaf3-4ea7-977e-51d9ff646b7e" containerName="maas-api" containerID="cri-o://2b373c88f1d76ab0d023400ee64b036ff512ed8aabc3d611261ed4d2c86a6856" gracePeriod=30 Oct 30 13:12:13.174918 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:13.174740 5029 generic.go:356] "Generic (PLEG): container finished" podID="5f73d872-eaf3-4ea7-977e-51d9ff646b7e" containerID="2b373c88f1d76ab0d023400ee64b036ff512ed8aabc3d611261ed4d2c86a6856" exitCode=0 Oct 30 13:12:13.175389 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:13.175330 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" event={"ID":"5f73d872-eaf3-4ea7-977e-51d9ff646b7e","Type":"ContainerDied","Data":"2b373c88f1d76ab0d023400ee64b036ff512ed8aabc3d611261ed4d2c86a6856"} Oct 30 13:12:13.231100 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:13.229178 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" Oct 30 13:12:13.300509 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:13.300478 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-kube-api-access-jf2cp\" (UniqueName: \"kubernetes.io/projected/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-kube-api-access-jf2cp\") pod \"5f73d872-eaf3-4ea7-977e-51d9ff646b7e\" (UID: \"5f73d872-eaf3-4ea7-977e-51d9ff646b7e\") " Oct 30 13:12:13.300706 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:13.300546 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-openshift-service-ca\") pod \"5f73d872-eaf3-4ea7-977e-51d9ff646b7e\" (UID: \"5f73d872-eaf3-4ea7-977e-51d9ff646b7e\") " Oct 30 13:12:13.300706 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:13.300580 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-maas-api-tls\") pod \"5f73d872-eaf3-4ea7-977e-51d9ff646b7e\" (UID: \"5f73d872-eaf3-4ea7-977e-51d9ff646b7e\") " Oct 30 13:12:13.300706 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:13.300617 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-metrics-certs\") pod \"5f73d872-eaf3-4ea7-977e-51d9ff646b7e\" (UID: \"5f73d872-eaf3-4ea7-977e-51d9ff646b7e\") " Oct 30 13:12:13.301196 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:13.301164 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-openshift-service-ca" pod "5f73d872-eaf3-4ea7-977e-51d9ff646b7e" (UID: "5f73d872-eaf3-4ea7-977e-51d9ff646b7e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:12:13.303569 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:13.303544 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-metrics-certs" pod "5f73d872-eaf3-4ea7-977e-51d9ff646b7e" (UID: "5f73d872-eaf3-4ea7-977e-51d9ff646b7e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:12:13.304417 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:13.304391 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-maas-api-tls" pod "5f73d872-eaf3-4ea7-977e-51d9ff646b7e" (UID: "5f73d872-eaf3-4ea7-977e-51d9ff646b7e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:12:13.304629 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:13.304608 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-kube-api-access-jf2cp" pod "5f73d872-eaf3-4ea7-977e-51d9ff646b7e" (UID: "5f73d872-eaf3-4ea7-977e-51d9ff646b7e"). InnerVolumeSpecName "kube-api-access-jf2cp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:12:13.401963 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:13.401882 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jf2cp\" (UniqueName: \"kubernetes.io/projected/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-kube-api-access-jf2cp\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:13.401963 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:13.401912 5029 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-openshift-service-ca\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:13.401963 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:13.401922 5029 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-maas-api-tls\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:13.401963 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:13.401942 5029 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f73d872-eaf3-4ea7-977e-51d9ff646b7e-metrics-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:14.188921 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:14.188824 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" event={"ID":"5f73d872-eaf3-4ea7-977e-51d9ff646b7e","Type":"ContainerDied","Data":"adcf12613a3db5fea75fbc56039f48fc523e24d2e42b934b8980cab1b7caf9ee"} Oct 30 13:12:14.189389 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:14.188939 5029 scope.go:122] "RemoveContainer" containerID="2b373c88f1d76ab0d023400ee64b036ff512ed8aabc3d611261ed4d2c86a6856" Oct 30 13:12:14.189389 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:14.189204 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" Oct 30 13:12:14.243307 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:14.243277 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x"] Oct 30 13:12:14.254284 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:14.254262 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x"] Oct 30 13:12:14.879672 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:12:14.879637 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:12:15.089313 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:15.089280 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5f73d872-eaf3-4ea7-977e-51d9ff646b7e" path="/var/lib/kubelet/pods/5f73d872-eaf3-4ea7-977e-51d9ff646b7e/volumes" Oct 30 13:12:16.112389 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:16.112346 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-484607-55747bf477-7pb4x" podUID="5f73d872-eaf3-4ea7-977e-51d9ff646b7e" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.49:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:12:21.896414 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:21.896384 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25"] Oct 30 13:12:21.896892 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:21.896643 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" podUID="6ef12288-2a06-4ad5-84ad-df2bcb9331d0" containerName="istio-proxy" containerID="cri-o://16a98e96090107e7caefd3a40cc770cedda99f43c802cd9c4a504b467223d92c" gracePeriod=30 Oct 30 13:12:21.909978 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:21.909936 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx"] Oct 30 13:12:21.910268 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:21.910216 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" podUID="88f14514-918e-43a1-a097-c236701d22f3" containerName="istio-proxy" containerID="cri-o://138ff646b89ea08b16c04bc1f6eb11a2f8063004998d271c0bb497c6bbcf8beb" gracePeriod=30 Oct 30 13:12:21.917113 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:21.917042 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg"] Oct 30 13:12:21.917387 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:21.917289 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" podUID="4f6573b8-e44b-4e54-9b3d-fba69a3dd375" containerName="istio-proxy" containerID="cri-o://37bcba6fec8dfe8eddeeafd8c6366d9f8b0dcec2e3e935bbcb8af7bd30a9ecb8" gracePeriod=30 Oct 30 13:12:23.275370 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:23.275336 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr"] Oct 30 13:12:23.285937 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:23.285900 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld"] Oct 30 13:12:23.293464 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:23.293439 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44"] Oct 30 13:12:27.206210 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.206126 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:12:27.228033 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.228011 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:12:27.244320 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.244299 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:12:27.267673 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.267621 5029 generic.go:356] "Generic (PLEG): container finished" podID="88f14514-918e-43a1-a097-c236701d22f3" containerID="138ff646b89ea08b16c04bc1f6eb11a2f8063004998d271c0bb497c6bbcf8beb" exitCode=0 Oct 30 13:12:27.267844 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.267737 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" event={"ID":"88f14514-918e-43a1-a097-c236701d22f3","Type":"ContainerDied","Data":"138ff646b89ea08b16c04bc1f6eb11a2f8063004998d271c0bb497c6bbcf8beb"} Oct 30 13:12:27.267844 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.267790 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" event={"ID":"88f14514-918e-43a1-a097-c236701d22f3","Type":"ContainerDied","Data":"d873aed3ca6d0ac5657846de971cc2abafd18dfbe2de88001fcab5399ad8edb3"} Oct 30 13:12:27.267844 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.267822 5029 scope.go:122] "RemoveContainer" containerID="138ff646b89ea08b16c04bc1f6eb11a2f8063004998d271c0bb497c6bbcf8beb" Oct 30 13:12:27.268028 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.267960 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx" Oct 30 13:12:27.270882 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.270852 5029 generic.go:356] "Generic (PLEG): container finished" podID="6ef12288-2a06-4ad5-84ad-df2bcb9331d0" containerID="16a98e96090107e7caefd3a40cc770cedda99f43c802cd9c4a504b467223d92c" exitCode=0 Oct 30 13:12:27.270998 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.270920 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" Oct 30 13:12:27.271185 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.270996 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" event={"ID":"6ef12288-2a06-4ad5-84ad-df2bcb9331d0","Type":"ContainerDied","Data":"16a98e96090107e7caefd3a40cc770cedda99f43c802cd9c4a504b467223d92c"} Oct 30 13:12:27.271185 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.271145 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25" event={"ID":"6ef12288-2a06-4ad5-84ad-df2bcb9331d0","Type":"ContainerDied","Data":"c91004fee529af2613c20f425f0ddd9ba73eb29672f207f2773ab25becee73c6"} Oct 30 13:12:27.273136 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.273112 5029 generic.go:356] "Generic (PLEG): container finished" podID="4f6573b8-e44b-4e54-9b3d-fba69a3dd375" containerID="37bcba6fec8dfe8eddeeafd8c6366d9f8b0dcec2e3e935bbcb8af7bd30a9ecb8" exitCode=0 Oct 30 13:12:27.273313 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.273281 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" event={"ID":"4f6573b8-e44b-4e54-9b3d-fba69a3dd375","Type":"ContainerDied","Data":"37bcba6fec8dfe8eddeeafd8c6366d9f8b0dcec2e3e935bbcb8af7bd30a9ecb8"} Oct 30 13:12:27.273389 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.273321 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" event={"ID":"4f6573b8-e44b-4e54-9b3d-fba69a3dd375","Type":"ContainerDied","Data":"dbd37aeb7777bd3163cd0a678ba5aee766c22a2b921d41a4f290171da2851551"} Oct 30 13:12:27.273389 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.273385 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg" Oct 30 13:12:27.282781 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.282759 5029 scope.go:122] "RemoveContainer" containerID="138ff646b89ea08b16c04bc1f6eb11a2f8063004998d271c0bb497c6bbcf8beb" Oct 30 13:12:27.283167 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:12:27.283145 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"138ff646b89ea08b16c04bc1f6eb11a2f8063004998d271c0bb497c6bbcf8beb\": container with ID starting with 138ff646b89ea08b16c04bc1f6eb11a2f8063004998d271c0bb497c6bbcf8beb not found: ID does not exist" containerID="138ff646b89ea08b16c04bc1f6eb11a2f8063004998d271c0bb497c6bbcf8beb" Oct 30 13:12:27.283265 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.283179 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"138ff646b89ea08b16c04bc1f6eb11a2f8063004998d271c0bb497c6bbcf8beb"} err="failed to get container status \"138ff646b89ea08b16c04bc1f6eb11a2f8063004998d271c0bb497c6bbcf8beb\": rpc error: code = NotFound desc = could not find container \"138ff646b89ea08b16c04bc1f6eb11a2f8063004998d271c0bb497c6bbcf8beb\": container with ID starting with 138ff646b89ea08b16c04bc1f6eb11a2f8063004998d271c0bb497c6bbcf8beb not found: ID does not exist" Oct 30 13:12:27.283265 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.283201 5029 scope.go:122] "RemoveContainer" containerID="16a98e96090107e7caefd3a40cc770cedda99f43c802cd9c4a504b467223d92c" Oct 30 13:12:27.300472 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.300450 5029 scope.go:122] "RemoveContainer" containerID="16a98e96090107e7caefd3a40cc770cedda99f43c802cd9c4a504b467223d92c" Oct 30 13:12:27.300767 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:12:27.300746 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16a98e96090107e7caefd3a40cc770cedda99f43c802cd9c4a504b467223d92c\": container with ID starting with 16a98e96090107e7caefd3a40cc770cedda99f43c802cd9c4a504b467223d92c not found: ID does not exist" containerID="16a98e96090107e7caefd3a40cc770cedda99f43c802cd9c4a504b467223d92c" Oct 30 13:12:27.300846 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.300777 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16a98e96090107e7caefd3a40cc770cedda99f43c802cd9c4a504b467223d92c"} err="failed to get container status \"16a98e96090107e7caefd3a40cc770cedda99f43c802cd9c4a504b467223d92c\": rpc error: code = NotFound desc = could not find container \"16a98e96090107e7caefd3a40cc770cedda99f43c802cd9c4a504b467223d92c\": container with ID starting with 16a98e96090107e7caefd3a40cc770cedda99f43c802cd9c4a504b467223d92c not found: ID does not exist" Oct 30 13:12:27.300846 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.300800 5029 scope.go:122] "RemoveContainer" containerID="37bcba6fec8dfe8eddeeafd8c6366d9f8b0dcec2e3e935bbcb8af7bd30a9ecb8" Oct 30 13:12:27.301087 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.301052 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-credential-socket\") pod \"88f14514-918e-43a1-a097-c236701d22f3\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " Oct 30 13:12:27.301174 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.301126 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/88f14514-918e-43a1-a097-c236701d22f3-istio-token\" (UniqueName: \"kubernetes.io/projected/88f14514-918e-43a1-a097-c236701d22f3-istio-token\") pod \"88f14514-918e-43a1-a097-c236701d22f3\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " Oct 30 13:12:27.301174 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.301155 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-podinfo\") pod \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " Oct 30 13:12:27.301291 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.301190 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-kube-api-access-dcv92\" (UniqueName: \"kubernetes.io/projected/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-kube-api-access-dcv92\") pod \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " Oct 30 13:12:27.301291 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.301231 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-data\") pod \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " Oct 30 13:12:27.301291 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.301270 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-workload-socket\") pod \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " Oct 30 13:12:27.301446 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.301320 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/88f14514-918e-43a1-a097-c236701d22f3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88f14514-918e-43a1-a097-c236701d22f3-istio-podinfo\") pod \"88f14514-918e-43a1-a097-c236701d22f3\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " Oct 30 13:12:27.301446 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.301357 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istiod-ca-cert\") pod \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " Oct 30 13:12:27.301446 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.301397 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-workload-socket\") pod \"88f14514-918e-43a1-a097-c236701d22f3\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " Oct 30 13:12:27.301599 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.301447 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-token\" (UniqueName: \"kubernetes.io/projected/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-token\") pod \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " Oct 30 13:12:27.301599 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.301563 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-credential-socket\") pod \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " Oct 30 13:12:27.301599 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.301578 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-data" pod "6ef12288-2a06-4ad5-84ad-df2bcb9331d0" (UID: "6ef12288-2a06-4ad5-84ad-df2bcb9331d0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:27.301757 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.301627 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-workload-certs\") pod \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " Oct 30 13:12:27.301757 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.301715 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/88f14514-918e-43a1-a097-c236701d22f3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88f14514-918e-43a1-a097-c236701d22f3-istiod-ca-cert\") pod \"88f14514-918e-43a1-a097-c236701d22f3\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " Oct 30 13:12:27.301757 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.301700 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-credential-socket" pod "88f14514-918e-43a1-a097-c236701d22f3" (UID: "88f14514-918e-43a1-a097-c236701d22f3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:27.301909 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.301753 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-istio-envoy\") pod \"88f14514-918e-43a1-a097-c236701d22f3\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " Oct 30 13:12:27.301909 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.301791 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/88f14514-918e-43a1-a097-c236701d22f3-kube-api-access-q22wf\" (UniqueName: \"kubernetes.io/projected/88f14514-918e-43a1-a097-c236701d22f3-kube-api-access-q22wf\") pod \"88f14514-918e-43a1-a097-c236701d22f3\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " Oct 30 13:12:27.301909 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.301828 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-envoy\") pod \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\" (UID: \"6ef12288-2a06-4ad5-84ad-df2bcb9331d0\") " Oct 30 13:12:27.301909 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.301873 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-workload-certs\") pod \"88f14514-918e-43a1-a097-c236701d22f3\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " Oct 30 13:12:27.301909 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.301899 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-istio-data\") pod \"88f14514-918e-43a1-a097-c236701d22f3\" (UID: \"88f14514-918e-43a1-a097-c236701d22f3\") " Oct 30 13:12:27.302176 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.301907 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istiod-ca-cert" pod "6ef12288-2a06-4ad5-84ad-df2bcb9331d0" (UID: "6ef12288-2a06-4ad5-84ad-df2bcb9331d0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:12:27.302176 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.302151 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-workload-certs" pod "6ef12288-2a06-4ad5-84ad-df2bcb9331d0" (UID: "6ef12288-2a06-4ad5-84ad-df2bcb9331d0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:27.302296 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.302167 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-workload-socket" pod "88f14514-918e-43a1-a097-c236701d22f3" (UID: "88f14514-918e-43a1-a097-c236701d22f3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:27.302296 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.302191 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-workload-socket" pod "6ef12288-2a06-4ad5-84ad-df2bcb9331d0" (UID: "6ef12288-2a06-4ad5-84ad-df2bcb9331d0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:27.302402 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.302372 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-credential-socket" pod "6ef12288-2a06-4ad5-84ad-df2bcb9331d0" (UID: "6ef12288-2a06-4ad5-84ad-df2bcb9331d0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:27.302656 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.302552 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-workload-certs" pod "88f14514-918e-43a1-a097-c236701d22f3" (UID: "88f14514-918e-43a1-a097-c236701d22f3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:27.302780 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.302677 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-istio-data" pod "88f14514-918e-43a1-a097-c236701d22f3" (UID: "88f14514-918e-43a1-a097-c236701d22f3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:27.302780 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.302711 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88f14514-918e-43a1-a097-c236701d22f3-istiod-ca-cert" pod "88f14514-918e-43a1-a097-c236701d22f3" (UID: "88f14514-918e-43a1-a097-c236701d22f3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:12:27.303214 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.303188 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-credential-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.303318 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.303219 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-workload-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.303318 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.303237 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/88f14514-918e-43a1-a097-c236701d22f3-istiod-ca-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.303318 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.303251 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-workload-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.303318 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.303264 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-istio-data\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.303318 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.303278 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-credential-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.303318 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.303292 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-data\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.303318 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.303303 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-workload-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.303318 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.303313 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istiod-ca-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.303754 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.303324 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-workload-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.304009 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.303969 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88f14514-918e-43a1-a097-c236701d22f3-istio-token" pod "88f14514-918e-43a1-a097-c236701d22f3" (UID: "88f14514-918e-43a1-a097-c236701d22f3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:12:27.304134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.304109 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-token" pod "6ef12288-2a06-4ad5-84ad-df2bcb9331d0" (UID: "6ef12288-2a06-4ad5-84ad-df2bcb9331d0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:12:27.304377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.304354 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-podinfo" pod "6ef12288-2a06-4ad5-84ad-df2bcb9331d0" (UID: "6ef12288-2a06-4ad5-84ad-df2bcb9331d0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 13:12:27.304460 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.304403 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/88f14514-918e-43a1-a097-c236701d22f3-istio-podinfo" pod "88f14514-918e-43a1-a097-c236701d22f3" (UID: "88f14514-918e-43a1-a097-c236701d22f3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 13:12:27.305323 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.305296 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88f14514-918e-43a1-a097-c236701d22f3-kube-api-access-q22wf" pod "88f14514-918e-43a1-a097-c236701d22f3" (UID: "88f14514-918e-43a1-a097-c236701d22f3"). InnerVolumeSpecName "kube-api-access-q22wf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:12:27.305901 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.305879 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-kube-api-access-dcv92" pod "6ef12288-2a06-4ad5-84ad-df2bcb9331d0" (UID: "6ef12288-2a06-4ad5-84ad-df2bcb9331d0"). InnerVolumeSpecName "kube-api-access-dcv92". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:12:27.305991 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.305938 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-istio-envoy" pod "88f14514-918e-43a1-a097-c236701d22f3" (UID: "88f14514-918e-43a1-a097-c236701d22f3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:27.306201 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.306184 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-envoy" pod "6ef12288-2a06-4ad5-84ad-df2bcb9331d0" (UID: "6ef12288-2a06-4ad5-84ad-df2bcb9331d0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:27.311749 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.311730 5029 scope.go:122] "RemoveContainer" containerID="37bcba6fec8dfe8eddeeafd8c6366d9f8b0dcec2e3e935bbcb8af7bd30a9ecb8" Oct 30 13:12:27.312023 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:12:27.312007 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37bcba6fec8dfe8eddeeafd8c6366d9f8b0dcec2e3e935bbcb8af7bd30a9ecb8\": container with ID starting with 37bcba6fec8dfe8eddeeafd8c6366d9f8b0dcec2e3e935bbcb8af7bd30a9ecb8 not found: ID does not exist" containerID="37bcba6fec8dfe8eddeeafd8c6366d9f8b0dcec2e3e935bbcb8af7bd30a9ecb8" Oct 30 13:12:27.312150 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.312040 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37bcba6fec8dfe8eddeeafd8c6366d9f8b0dcec2e3e935bbcb8af7bd30a9ecb8"} err="failed to get container status \"37bcba6fec8dfe8eddeeafd8c6366d9f8b0dcec2e3e935bbcb8af7bd30a9ecb8\": rpc error: code = NotFound desc = could not find container \"37bcba6fec8dfe8eddeeafd8c6366d9f8b0dcec2e3e935bbcb8af7bd30a9ecb8\": container with ID starting with 37bcba6fec8dfe8eddeeafd8c6366d9f8b0dcec2e3e935bbcb8af7bd30a9ecb8 not found: ID does not exist" Oct 30 13:12:27.404453 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.404419 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istiod-ca-cert\") pod \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " Oct 30 13:12:27.404647 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.404464 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-workload-certs\") pod \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " Oct 30 13:12:27.404647 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.404501 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-workload-socket\") pod \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " Oct 30 13:12:27.404647 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.404524 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-token\" (UniqueName: \"kubernetes.io/projected/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-token\") pod \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " Oct 30 13:12:27.404647 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.404572 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-data\") pod \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " Oct 30 13:12:27.404647 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.404613 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-podinfo\") pod \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " Oct 30 13:12:27.404647 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.404649 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-kube-api-access-tf2kv\" (UniqueName: \"kubernetes.io/projected/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-kube-api-access-tf2kv\") pod \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " Oct 30 13:12:27.404971 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.404670 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-envoy\") pod \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " Oct 30 13:12:27.404971 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.404709 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-workload-certs" pod "4f6573b8-e44b-4e54-9b3d-fba69a3dd375" (UID: "4f6573b8-e44b-4e54-9b3d-fba69a3dd375"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:27.404971 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.404737 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-credential-socket\") pod \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\" (UID: \"4f6573b8-e44b-4e54-9b3d-fba69a3dd375\") " Oct 30 13:12:27.404971 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.404851 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-workload-socket" pod "4f6573b8-e44b-4e54-9b3d-fba69a3dd375" (UID: "4f6573b8-e44b-4e54-9b3d-fba69a3dd375"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:27.404971 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.404920 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istiod-ca-cert" pod "4f6573b8-e44b-4e54-9b3d-fba69a3dd375" (UID: "4f6573b8-e44b-4e54-9b3d-fba69a3dd375"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:12:27.405286 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.405111 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-credential-socket" pod "4f6573b8-e44b-4e54-9b3d-fba69a3dd375" (UID: "4f6573b8-e44b-4e54-9b3d-fba69a3dd375"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:27.405286 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.405222 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-data" pod "4f6573b8-e44b-4e54-9b3d-fba69a3dd375" (UID: "4f6573b8-e44b-4e54-9b3d-fba69a3dd375"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:27.405400 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.405352 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-credential-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.405400 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.405367 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/88f14514-918e-43a1-a097-c236701d22f3-istio-envoy\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.405400 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.405377 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q22wf\" (UniqueName: \"kubernetes.io/projected/88f14514-918e-43a1-a097-c236701d22f3-kube-api-access-q22wf\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.405400 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.405386 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-envoy\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.405400 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.405395 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88f14514-918e-43a1-a097-c236701d22f3-istio-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.405400 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.405403 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-podinfo\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.405703 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.405411 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istiod-ca-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.405703 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.405419 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcv92\" (UniqueName: \"kubernetes.io/projected/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-kube-api-access-dcv92\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.405703 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.405427 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-workload-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.405703 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.405435 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-workload-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.405703 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.405442 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/88f14514-918e-43a1-a097-c236701d22f3-istio-podinfo\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.405703 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.405450 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-data\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.405703 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.405458 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ef12288-2a06-4ad5-84ad-df2bcb9331d0-istio-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.406802 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.406779 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-envoy" pod "4f6573b8-e44b-4e54-9b3d-fba69a3dd375" (UID: "4f6573b8-e44b-4e54-9b3d-fba69a3dd375"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:27.407059 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.407032 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-token" pod "4f6573b8-e44b-4e54-9b3d-fba69a3dd375" (UID: "4f6573b8-e44b-4e54-9b3d-fba69a3dd375"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:12:27.407279 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.407262 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-kube-api-access-tf2kv" pod "4f6573b8-e44b-4e54-9b3d-fba69a3dd375" (UID: "4f6573b8-e44b-4e54-9b3d-fba69a3dd375"). InnerVolumeSpecName "kube-api-access-tf2kv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:12:27.407721 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.407696 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-podinfo" pod "4f6573b8-e44b-4e54-9b3d-fba69a3dd375" (UID: "4f6573b8-e44b-4e54-9b3d-fba69a3dd375"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 13:12:27.506576 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.506504 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.506576 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.506531 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-podinfo\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.506576 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.506543 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tf2kv\" (UniqueName: \"kubernetes.io/projected/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-kube-api-access-tf2kv\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.506576 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.506553 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f6573b8-e44b-4e54-9b3d-fba69a3dd375-istio-envoy\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:27.619549 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.619511 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx"] Oct 30 13:12:27.629126 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.628429 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-gtfgx"] Oct 30 13:12:27.667114 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.667061 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25"] Oct 30 13:12:27.670562 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.670539 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-zbp25"] Oct 30 13:12:27.708511 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.708465 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg"] Oct 30 13:12:27.716824 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:27.716796 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-484607-openshift-default-b8b957995-7pdcg"] Oct 30 13:12:29.091839 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:29.091808 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4f6573b8-e44b-4e54-9b3d-fba69a3dd375" path="/var/lib/kubelet/pods/4f6573b8-e44b-4e54-9b3d-fba69a3dd375/volumes" Oct 30 13:12:29.092553 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:29.092529 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ef12288-2a06-4ad5-84ad-df2bcb9331d0" path="/var/lib/kubelet/pods/6ef12288-2a06-4ad5-84ad-df2bcb9331d0/volumes" Oct 30 13:12:29.093057 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:29.093036 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="88f14514-918e-43a1-a097-c236701d22f3" path="/var/lib/kubelet/pods/88f14514-918e-43a1-a097-c236701d22f3/volumes" Oct 30 13:12:35.709763 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.709726 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg"] Oct 30 13:12:35.710357 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710337 5029 cpu_manager.go:414] "containerMap: removing container" podUID="3447ed0f-ef06-4a75-8f49-67c043cf08cb" containerName="istio-proxy" Oct 30 13:12:35.710433 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710359 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3447ed0f-ef06-4a75-8f49-67c043cf08cb" containerName="istio-proxy" Oct 30 13:12:35.710433 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710379 5029 cpu_manager.go:414] "containerMap: removing container" podUID="0e89ac39-4946-4c8a-bbc0-43c2f8487aec" containerName="istio-proxy" Oct 30 13:12:35.710433 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710387 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e89ac39-4946-4c8a-bbc0-43c2f8487aec" containerName="istio-proxy" Oct 30 13:12:35.710433 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710401 5029 cpu_manager.go:414] "containerMap: removing container" podUID="6ef12288-2a06-4ad5-84ad-df2bcb9331d0" containerName="istio-proxy" Oct 30 13:12:35.710433 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710409 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ef12288-2a06-4ad5-84ad-df2bcb9331d0" containerName="istio-proxy" Oct 30 13:12:35.710433 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710428 5029 cpu_manager.go:414] "containerMap: removing container" podUID="184f5837-5aee-46b3-856c-4c3db17a7877" containerName="istio-proxy" Oct 30 13:12:35.710433 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710435 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="184f5837-5aee-46b3-856c-4c3db17a7877" containerName="istio-proxy" Oct 30 13:12:35.710668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710449 5029 cpu_manager.go:414] "containerMap: removing container" podUID="b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" containerName="main" Oct 30 13:12:35.710668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710455 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" containerName="main" Oct 30 13:12:35.710668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710468 5029 cpu_manager.go:414] "containerMap: removing container" podUID="6962e138-faa9-438c-9138-1133bdce1011" containerName="main" Oct 30 13:12:35.710668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710476 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6962e138-faa9-438c-9138-1133bdce1011" containerName="main" Oct 30 13:12:35.710668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710490 5029 cpu_manager.go:414] "containerMap: removing container" podUID="b80d9479-2325-40f0-93fd-329c462b629c" containerName="istio-proxy" Oct 30 13:12:35.710668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710497 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b80d9479-2325-40f0-93fd-329c462b629c" containerName="istio-proxy" Oct 30 13:12:35.710668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710527 5029 cpu_manager.go:414] "containerMap: removing container" podUID="9af7cbdb-5aee-4f11-9446-fceacd3fb135" containerName="main" Oct 30 13:12:35.710668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710535 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9af7cbdb-5aee-4f11-9446-fceacd3fb135" containerName="main" Oct 30 13:12:35.710668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710548 5029 cpu_manager.go:414] "containerMap: removing container" podUID="2080a9e8-70bc-4c2e-8b67-7bb00077837b" containerName="maas-api" Oct 30 13:12:35.710668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710556 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2080a9e8-70bc-4c2e-8b67-7bb00077837b" containerName="maas-api" Oct 30 13:12:35.710668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710570 5029 cpu_manager.go:414] "containerMap: removing container" podUID="5f73d872-eaf3-4ea7-977e-51d9ff646b7e" containerName="maas-api" Oct 30 13:12:35.710668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710576 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f73d872-eaf3-4ea7-977e-51d9ff646b7e" containerName="maas-api" Oct 30 13:12:35.710668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710586 5029 cpu_manager.go:414] "containerMap: removing container" podUID="617de128-a9b7-428c-acec-3753566fba40" containerName="istio-proxy" Oct 30 13:12:35.710668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710593 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="617de128-a9b7-428c-acec-3753566fba40" containerName="istio-proxy" Oct 30 13:12:35.710668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710612 5029 cpu_manager.go:414] "containerMap: removing container" podUID="fd2c4ae1-fbe5-46c9-82c7-f4f418994e31" containerName="istio-proxy" Oct 30 13:12:35.710668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710619 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fd2c4ae1-fbe5-46c9-82c7-f4f418994e31" containerName="istio-proxy" Oct 30 13:12:35.710668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710631 5029 cpu_manager.go:414] "containerMap: removing container" podUID="7717496d-5ae0-421f-80c8-b89810a6b1c6" containerName="maas-api" Oct 30 13:12:35.710668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710638 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7717496d-5ae0-421f-80c8-b89810a6b1c6" containerName="maas-api" Oct 30 13:12:35.710668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710652 5029 cpu_manager.go:414] "containerMap: removing container" podUID="4f6573b8-e44b-4e54-9b3d-fba69a3dd375" containerName="istio-proxy" Oct 30 13:12:35.710668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710658 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4f6573b8-e44b-4e54-9b3d-fba69a3dd375" containerName="istio-proxy" Oct 30 13:12:35.710668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710667 5029 cpu_manager.go:414] "containerMap: removing container" podUID="935c23b1-953a-40cc-ad0c-bec2ef61afba" containerName="main" Oct 30 13:12:35.710668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710673 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="935c23b1-953a-40cc-ad0c-bec2ef61afba" containerName="main" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710684 5029 cpu_manager.go:414] "containerMap: removing container" podUID="6328797e-74fc-4877-acac-0554466d541a" containerName="istio-proxy" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710690 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6328797e-74fc-4877-acac-0554466d541a" containerName="istio-proxy" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710699 5029 cpu_manager.go:414] "containerMap: removing container" podUID="ac96e53b-9dec-4b7a-bd58-833239583b83" containerName="istio-proxy" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710705 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac96e53b-9dec-4b7a-bd58-833239583b83" containerName="istio-proxy" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710716 5029 cpu_manager.go:414] "containerMap: removing container" podUID="88f14514-918e-43a1-a097-c236701d22f3" containerName="istio-proxy" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710722 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="88f14514-918e-43a1-a097-c236701d22f3" containerName="istio-proxy" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710853 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="935c23b1-953a-40cc-ad0c-bec2ef61afba" containerName="main" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710866 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="2080a9e8-70bc-4c2e-8b67-7bb00077837b" containerName="maas-api" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710874 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="3447ed0f-ef06-4a75-8f49-67c043cf08cb" containerName="istio-proxy" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710881 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e89ac39-4946-4c8a-bbc0-43c2f8487aec" containerName="istio-proxy" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710888 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f73d872-eaf3-4ea7-977e-51d9ff646b7e" containerName="maas-api" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710893 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ef12288-2a06-4ad5-84ad-df2bcb9331d0" containerName="istio-proxy" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710898 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="617de128-a9b7-428c-acec-3753566fba40" containerName="istio-proxy" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710902 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="9af7cbdb-5aee-4f11-9446-fceacd3fb135" containerName="main" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710909 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="6328797e-74fc-4877-acac-0554466d541a" containerName="istio-proxy" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710914 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac96e53b-9dec-4b7a-bd58-833239583b83" containerName="istio-proxy" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710921 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="fd2c4ae1-fbe5-46c9-82c7-f4f418994e31" containerName="istio-proxy" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710927 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="7717496d-5ae0-421f-80c8-b89810a6b1c6" containerName="maas-api" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710933 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="184f5837-5aee-46b3-856c-4c3db17a7877" containerName="istio-proxy" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710938 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6ce39f3-c5ad-4c48-9a01-84123c6c7ab2" containerName="main" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710943 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="4f6573b8-e44b-4e54-9b3d-fba69a3dd375" containerName="istio-proxy" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710949 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="6962e138-faa9-438c-9138-1133bdce1011" containerName="main" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710955 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="88f14514-918e-43a1-a097-c236701d22f3" containerName="istio-proxy" Oct 30 13:12:35.711377 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.710961 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="b80d9479-2325-40f0-93fd-329c462b629c" containerName="istio-proxy" Oct 30 13:12:35.715589 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.715566 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.731348 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.731325 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-ac0c58-openshift-default-dockercfg-5mbd5\"" Oct 30 13:12:35.744998 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.744974 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg"] Oct 30 13:12:35.807935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.807897 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-data\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.807935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.807935 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqf4m\" (UniqueName: \"kubernetes.io/projected/0840bdc7-256c-4b2b-a76a-8d5063993d00-kube-api-access-pqf4m\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.808170 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.808092 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-token\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.808170 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.808136 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-envoy\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.808301 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.808278 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-podinfo\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.808344 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.808329 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-workload-certs\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.808401 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.808386 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-workload-socket\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.808463 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.808450 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0840bdc7-256c-4b2b-a76a-8d5063993d00-istiod-ca-cert\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.808521 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.808508 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-credential-socket\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.909419 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.909385 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-token\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.909579 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.909429 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-envoy\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.909579 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.909481 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-podinfo\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.909579 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.909506 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-workload-certs\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.909692 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.909632 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-workload-socket\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.909766 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.909751 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0840bdc7-256c-4b2b-a76a-8d5063993d00-istiod-ca-cert\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.909806 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.909789 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-credential-socket\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.909858 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.909848 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-data\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.909906 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.909878 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqf4m\" (UniqueName: \"kubernetes.io/projected/0840bdc7-256c-4b2b-a76a-8d5063993d00-kube-api-access-pqf4m\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.909970 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.909944 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-workload-certs\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.910023 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.910005 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-workload-socket\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.910283 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.910257 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-credential-socket\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.910394 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.910308 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-data\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.910685 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.910660 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0840bdc7-256c-4b2b-a76a-8d5063993d00-istiod-ca-cert\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.911909 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.911884 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-envoy\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.912322 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.912296 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-podinfo\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.933371 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.933341 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-token\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:35.933454 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:35.933353 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqf4m\" (UniqueName: \"kubernetes.io/projected/0840bdc7-256c-4b2b-a76a-8d5063993d00-kube-api-access-pqf4m\") pod \"e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:36.029279 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:36.029199 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:36.357943 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:36.357817 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg"] Oct 30 13:12:36.360926 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:12:36.360899 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0840bdc7_256c_4b2b_a76a_8d5063993d00.slice/crio-beb3ff2c2bb791f13252a50162bebc92b49d8fc77984274d7bc6db6ec86f4027 WatchSource:0}: Error finding container beb3ff2c2bb791f13252a50162bebc92b49d8fc77984274d7bc6db6ec86f4027: Status 404 returned error can't find the container with id beb3ff2c2bb791f13252a50162bebc92b49d8fc77984274d7bc6db6ec86f4027 Oct 30 13:12:36.363057 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:36.363021 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:12:36.363164 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:36.363104 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:12:36.363164 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:36.363142 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:12:37.339101 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:37.339048 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" event={"ID":"0840bdc7-256c-4b2b-a76a-8d5063993d00","Type":"ContainerStarted","Data":"aa8578681b9f810eeb6f70b9f958a91120b989554a9853c1fbe8a3626cee557b"} Oct 30 13:12:37.339523 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:37.339121 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" event={"ID":"0840bdc7-256c-4b2b-a76a-8d5063993d00","Type":"ContainerStarted","Data":"beb3ff2c2bb791f13252a50162bebc92b49d8fc77984274d7bc6db6ec86f4027"} Oct 30 13:12:37.401322 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:37.401253 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" podStartSLOduration=2.4012327669999998 podStartE2EDuration="2.401232767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:12:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:12:37.394830932 +0000 UTC m=+1256.420472174" watchObservedRunningTime="2026-10-30 13:12:37.401232767 +0000 UTC m=+1256.426874019" Oct 30 13:12:38.030292 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.030256 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:38.031958 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.031928 5029 patch_prober.go:29] interesting pod/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Oct 30 13:12:38.032126 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.031995 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" podUID="0840bdc7-256c-4b2b-a76a-8d5063993d00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Oct 30 13:12:38.313936 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.313884 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" podUID="06d7d761-d47f-4aa5-b931-46f910b65597" containerName="main" containerID="cri-o://a65ea87c5d1877ced1ecac3014756daedd938caf6c1628f818ff8573deac55ba" gracePeriod=45 Oct 30 13:12:38.316766 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.316716 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" podUID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" containerName="main" containerID="cri-o://00749907ad1885b7e1665dff6f3ad94d2dd244caa067accdd9854339d7030b08" gracePeriod=45 Oct 30 13:12:38.316995 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.316720 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" podUID="2b878b27-465c-433d-aba4-e31849cfc14c" containerName="main" containerID="cri-o://2078ff625415e5439f8e1f3518b20846154f80f5d309076e148861d88a25c4d8" gracePeriod=45 Oct 30 13:12:38.325968 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.325941 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" podUID="06d7d761-d47f-4aa5-b931-46f910b65597" containerName="main" probeResult="failure" output="Get \"https://10.132.0.83:8000/ready\": EOF" Oct 30 13:12:38.712302 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.712274 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:12:38.732842 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.732818 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:12:38.759342 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.759197 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-dshm\") pod \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " Oct 30 13:12:38.759342 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.759246 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06d7d761-d47f-4aa5-b931-46f910b65597-kube-api-access-4q4cr\" (UniqueName: \"kubernetes.io/projected/06d7d761-d47f-4aa5-b931-46f910b65597-kube-api-access-4q4cr\") pod \"06d7d761-d47f-4aa5-b931-46f910b65597\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " Oct 30 13:12:38.759502 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.759350 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/06d7d761-d47f-4aa5-b931-46f910b65597-tls-certs\" (UniqueName: \"kubernetes.io/secret/06d7d761-d47f-4aa5-b931-46f910b65597-tls-certs\") pod \"06d7d761-d47f-4aa5-b931-46f910b65597\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " Oct 30 13:12:38.759502 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.759448 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-home\" (UniqueName: \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-home\") pod \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " Oct 30 13:12:38.759502 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.759492 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-model-cache\") pod \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " Oct 30 13:12:38.759603 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.759523 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-model-cache\") pod \"06d7d761-d47f-4aa5-b931-46f910b65597\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " Oct 30 13:12:38.759863 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.759841 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-home\" (UniqueName: \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-home\") pod \"06d7d761-d47f-4aa5-b931-46f910b65597\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " Oct 30 13:12:38.759940 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.759920 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-dshm\" (UniqueName: \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-dshm\") pod \"06d7d761-d47f-4aa5-b931-46f910b65597\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " Oct 30 13:12:38.759988 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.759949 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-tmp-dir\") pod \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " Oct 30 13:12:38.760038 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.760013 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-tls-certs\" (UniqueName: \"kubernetes.io/secret/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-tls-certs\") pod \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " Oct 30 13:12:38.760133 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.760096 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-kube-api-access-8jrhw\" (UniqueName: \"kubernetes.io/projected/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-kube-api-access-8jrhw\") pod \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\" (UID: \"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d\") " Oct 30 13:12:38.760192 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.760163 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-tmp-dir\") pod \"06d7d761-d47f-4aa5-b931-46f910b65597\" (UID: \"06d7d761-d47f-4aa5-b931-46f910b65597\") " Oct 30 13:12:38.760894 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.760867 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-tmp-dir" pod "06d7d761-d47f-4aa5-b931-46f910b65597" (UID: "06d7d761-d47f-4aa5-b931-46f910b65597"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:38.761060 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.761043 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-home" pod "cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" (UID: "cd68fdee-30c0-42ac-bf6e-aff5d2abb46d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:38.762045 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.761759 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-home" pod "06d7d761-d47f-4aa5-b931-46f910b65597" (UID: "06d7d761-d47f-4aa5-b931-46f910b65597"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:38.762179 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.762059 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-model-cache" pod "cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" (UID: "cd68fdee-30c0-42ac-bf6e-aff5d2abb46d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:38.762743 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.762472 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-model-cache" pod "06d7d761-d47f-4aa5-b931-46f910b65597" (UID: "06d7d761-d47f-4aa5-b931-46f910b65597"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:38.762882 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.762827 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-tmp-dir" pod "cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" (UID: "cd68fdee-30c0-42ac-bf6e-aff5d2abb46d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:38.765160 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.765125 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06d7d761-d47f-4aa5-b931-46f910b65597-kube-api-access-4q4cr" pod "06d7d761-d47f-4aa5-b931-46f910b65597" (UID: "06d7d761-d47f-4aa5-b931-46f910b65597"). InnerVolumeSpecName "kube-api-access-4q4cr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:12:38.766668 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.766648 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:12:38.789279 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.789213 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-dshm" pod "cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" (UID: "cd68fdee-30c0-42ac-bf6e-aff5d2abb46d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:38.789593 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.789560 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-dshm" pod "06d7d761-d47f-4aa5-b931-46f910b65597" (UID: "06d7d761-d47f-4aa5-b931-46f910b65597"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:38.789835 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.789805 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06d7d761-d47f-4aa5-b931-46f910b65597-tls-certs" pod "06d7d761-d47f-4aa5-b931-46f910b65597" (UID: "06d7d761-d47f-4aa5-b931-46f910b65597"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:12:38.791959 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.791922 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-kube-api-access-8jrhw" pod "cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" (UID: "cd68fdee-30c0-42ac-bf6e-aff5d2abb46d"). InnerVolumeSpecName "kube-api-access-8jrhw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:12:38.813534 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.813494 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-tls-certs" pod "cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" (UID: "cd68fdee-30c0-42ac-bf6e-aff5d2abb46d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:12:38.861121 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.861050 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-tmp-dir\") pod \"2b878b27-465c-433d-aba4-e31849cfc14c\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " Oct 30 13:12:38.861306 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.861146 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-home\" (UniqueName: \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-home\") pod \"2b878b27-465c-433d-aba4-e31849cfc14c\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " Oct 30 13:12:38.861306 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.861208 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-model-cache\") pod \"2b878b27-465c-433d-aba4-e31849cfc14c\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " Oct 30 13:12:38.861306 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.861267 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-dshm\") pod \"2b878b27-465c-433d-aba4-e31849cfc14c\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " Oct 30 13:12:38.861306 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.861296 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2b878b27-465c-433d-aba4-e31849cfc14c-tls-certs\" (UniqueName: \"kubernetes.io/secret/2b878b27-465c-433d-aba4-e31849cfc14c-tls-certs\") pod \"2b878b27-465c-433d-aba4-e31849cfc14c\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " Oct 30 13:12:38.861495 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.861370 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b878b27-465c-433d-aba4-e31849cfc14c-kube-api-access-lmpj9\" (UniqueName: \"kubernetes.io/projected/2b878b27-465c-433d-aba4-e31849cfc14c-kube-api-access-lmpj9\") pod \"2b878b27-465c-433d-aba4-e31849cfc14c\" (UID: \"2b878b27-465c-433d-aba4-e31849cfc14c\") " Oct 30 13:12:38.861804 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.861785 5029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06d7d761-d47f-4aa5-b931-46f910b65597-tls-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:38.861886 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.861806 5029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-home\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:38.861886 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.861818 5029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-model-cache\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:38.861886 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.861830 5029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-model-cache\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:38.861886 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.861841 5029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-home\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:38.861886 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.861856 5029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-dshm\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:38.861886 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.861870 5029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-tmp-dir\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:38.861886 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.861884 5029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-tls-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:38.862236 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.861899 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jrhw\" (UniqueName: \"kubernetes.io/projected/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-kube-api-access-8jrhw\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:38.862236 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.861912 5029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06d7d761-d47f-4aa5-b931-46f910b65597-tmp-dir\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:38.862236 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.861935 5029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d-dshm\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:38.862236 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.861949 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4q4cr\" (UniqueName: \"kubernetes.io/projected/06d7d761-d47f-4aa5-b931-46f910b65597-kube-api-access-4q4cr\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:38.863557 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.863527 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-model-cache" pod "2b878b27-465c-433d-aba4-e31849cfc14c" (UID: "2b878b27-465c-433d-aba4-e31849cfc14c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:38.863690 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.863622 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-home" pod "2b878b27-465c-433d-aba4-e31849cfc14c" (UID: "2b878b27-465c-433d-aba4-e31849cfc14c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:38.863690 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.863659 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-tmp-dir" pod "2b878b27-465c-433d-aba4-e31849cfc14c" (UID: "2b878b27-465c-433d-aba4-e31849cfc14c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:38.864728 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.864493 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b878b27-465c-433d-aba4-e31849cfc14c-kube-api-access-lmpj9" pod "2b878b27-465c-433d-aba4-e31849cfc14c" (UID: "2b878b27-465c-433d-aba4-e31849cfc14c"). InnerVolumeSpecName "kube-api-access-lmpj9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:12:38.865416 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.865394 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-dshm" pod "2b878b27-465c-433d-aba4-e31849cfc14c" (UID: "2b878b27-465c-433d-aba4-e31849cfc14c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:12:38.866634 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.866606 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b878b27-465c-433d-aba4-e31849cfc14c-tls-certs" pod "2b878b27-465c-433d-aba4-e31849cfc14c" (UID: "2b878b27-465c-433d-aba4-e31849cfc14c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:12:38.963321 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.963249 5029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-tmp-dir\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:38.963321 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.963276 5029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-home\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:38.963321 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.963287 5029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-model-cache\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:38.963321 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.963294 5029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b878b27-465c-433d-aba4-e31849cfc14c-dshm\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:38.963321 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.963302 5029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b878b27-465c-433d-aba4-e31849cfc14c-tls-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:38.963321 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:38.963311 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmpj9\" (UniqueName: \"kubernetes.io/projected/2b878b27-465c-433d-aba4-e31849cfc14c-kube-api-access-lmpj9\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:39.030516 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.030482 5029 patch_prober.go:29] interesting pod/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Oct 30 13:12:39.030687 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.030537 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" podUID="0840bdc7-256c-4b2b-a76a-8d5063993d00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Oct 30 13:12:39.354609 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.354577 5029 generic.go:356] "Generic (PLEG): container finished" podID="06d7d761-d47f-4aa5-b931-46f910b65597" containerID="a65ea87c5d1877ced1ecac3014756daedd938caf6c1628f818ff8573deac55ba" exitCode=0 Oct 30 13:12:39.354786 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.354618 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" Oct 30 13:12:39.354786 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.354666 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" event={"ID":"06d7d761-d47f-4aa5-b931-46f910b65597","Type":"ContainerDied","Data":"a65ea87c5d1877ced1ecac3014756daedd938caf6c1628f818ff8573deac55ba"} Oct 30 13:12:39.354786 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.354705 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr" event={"ID":"06d7d761-d47f-4aa5-b931-46f910b65597","Type":"ContainerDied","Data":"e397a0c944576f2074bc739223c5335638af23c6d034adc10e713a89abf7cf3f"} Oct 30 13:12:39.354786 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.354724 5029 scope.go:122] "RemoveContainer" containerID="a65ea87c5d1877ced1ecac3014756daedd938caf6c1628f818ff8573deac55ba" Oct 30 13:12:39.356411 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.356391 5029 generic.go:356] "Generic (PLEG): container finished" podID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" containerID="00749907ad1885b7e1665dff6f3ad94d2dd244caa067accdd9854339d7030b08" exitCode=0 Oct 30 13:12:39.356501 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.356430 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" Oct 30 13:12:39.356501 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.356482 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" event={"ID":"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d","Type":"ContainerDied","Data":"00749907ad1885b7e1665dff6f3ad94d2dd244caa067accdd9854339d7030b08"} Oct 30 13:12:39.356587 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.356529 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44" event={"ID":"cd68fdee-30c0-42ac-bf6e-aff5d2abb46d","Type":"ContainerDied","Data":"406bdfdffb1172cf9fb633674b9c5e1a74d8430a21c3a88d084dcfdb50cba135"} Oct 30 13:12:39.358247 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.358228 5029 generic.go:356] "Generic (PLEG): container finished" podID="2b878b27-465c-433d-aba4-e31849cfc14c" containerID="2078ff625415e5439f8e1f3518b20846154f80f5d309076e148861d88a25c4d8" exitCode=0 Oct 30 13:12:39.358358 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.358253 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" event={"ID":"2b878b27-465c-433d-aba4-e31849cfc14c","Type":"ContainerDied","Data":"2078ff625415e5439f8e1f3518b20846154f80f5d309076e148861d88a25c4d8"} Oct 30 13:12:39.358358 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.358283 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" Oct 30 13:12:39.358454 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.358288 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld" event={"ID":"2b878b27-465c-433d-aba4-e31849cfc14c","Type":"ContainerDied","Data":"cf4bdb1974d25104f6f24eff9333caab9758e77ef376b6df2a5b8b037f1fafba"} Oct 30 13:12:39.365707 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.365691 5029 scope.go:122] "RemoveContainer" containerID="a65ea87c5d1877ced1ecac3014756daedd938caf6c1628f818ff8573deac55ba" Oct 30 13:12:39.365976 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:12:39.365952 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a65ea87c5d1877ced1ecac3014756daedd938caf6c1628f818ff8573deac55ba\": container with ID starting with a65ea87c5d1877ced1ecac3014756daedd938caf6c1628f818ff8573deac55ba not found: ID does not exist" containerID="a65ea87c5d1877ced1ecac3014756daedd938caf6c1628f818ff8573deac55ba" Oct 30 13:12:39.366108 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.365981 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a65ea87c5d1877ced1ecac3014756daedd938caf6c1628f818ff8573deac55ba"} err="failed to get container status \"a65ea87c5d1877ced1ecac3014756daedd938caf6c1628f818ff8573deac55ba\": rpc error: code = NotFound desc = could not find container \"a65ea87c5d1877ced1ecac3014756daedd938caf6c1628f818ff8573deac55ba\": container with ID starting with a65ea87c5d1877ced1ecac3014756daedd938caf6c1628f818ff8573deac55ba not found: ID does not exist" Oct 30 13:12:39.366108 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.365998 5029 scope.go:122] "RemoveContainer" containerID="00749907ad1885b7e1665dff6f3ad94d2dd244caa067accdd9854339d7030b08" Oct 30 13:12:39.405510 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.405482 5029 scope.go:122] "RemoveContainer" containerID="00749907ad1885b7e1665dff6f3ad94d2dd244caa067accdd9854339d7030b08" Oct 30 13:12:39.405886 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:12:39.405864 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00749907ad1885b7e1665dff6f3ad94d2dd244caa067accdd9854339d7030b08\": container with ID starting with 00749907ad1885b7e1665dff6f3ad94d2dd244caa067accdd9854339d7030b08 not found: ID does not exist" containerID="00749907ad1885b7e1665dff6f3ad94d2dd244caa067accdd9854339d7030b08" Oct 30 13:12:39.405997 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.405898 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00749907ad1885b7e1665dff6f3ad94d2dd244caa067accdd9854339d7030b08"} err="failed to get container status \"00749907ad1885b7e1665dff6f3ad94d2dd244caa067accdd9854339d7030b08\": rpc error: code = NotFound desc = could not find container \"00749907ad1885b7e1665dff6f3ad94d2dd244caa067accdd9854339d7030b08\": container with ID starting with 00749907ad1885b7e1665dff6f3ad94d2dd244caa067accdd9854339d7030b08 not found: ID does not exist" Oct 30 13:12:39.405997 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.405929 5029 scope.go:122] "RemoveContainer" containerID="2078ff625415e5439f8e1f3518b20846154f80f5d309076e148861d88a25c4d8" Oct 30 13:12:39.407873 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.407845 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld"] Oct 30 13:12:39.415706 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.415681 5029 scope.go:122] "RemoveContainer" containerID="2078ff625415e5439f8e1f3518b20846154f80f5d309076e148861d88a25c4d8" Oct 30 13:12:39.416017 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:12:39.415989 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2078ff625415e5439f8e1f3518b20846154f80f5d309076e148861d88a25c4d8\": container with ID starting with 2078ff625415e5439f8e1f3518b20846154f80f5d309076e148861d88a25c4d8 not found: ID does not exist" containerID="2078ff625415e5439f8e1f3518b20846154f80f5d309076e148861d88a25c4d8" Oct 30 13:12:39.416116 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.416017 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2078ff625415e5439f8e1f3518b20846154f80f5d309076e148861d88a25c4d8"} err="failed to get container status \"2078ff625415e5439f8e1f3518b20846154f80f5d309076e148861d88a25c4d8\": rpc error: code = NotFound desc = could not find container \"2078ff625415e5439f8e1f3518b20846154f80f5d309076e148861d88a25c4d8\": container with ID starting with 2078ff625415e5439f8e1f3518b20846154f80f5d309076e148861d88a25c4d8 not found: ID does not exist" Oct 30 13:12:39.418577 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.418557 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-ac0c58/premium-w1-ac0c58-kserve-557f9bcc9c-vqzld"] Oct 30 13:12:39.446139 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.446114 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr"] Oct 30 13:12:39.449048 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.449028 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-ac0c58/distinct-w1-ac0c58-kserve-dccc84448-zjdfr"] Oct 30 13:12:39.498616 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.498577 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44"] Oct 30 13:12:39.501754 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:39.501727 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-ac0c58/simulator-w1-ac0c58-kserve-74b44db96d-rrz44"] Oct 30 13:12:40.029851 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:40.029818 5029 patch_prober.go:29] interesting pod/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Oct 30 13:12:40.030263 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:40.029890 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" podUID="0840bdc7-256c-4b2b-a76a-8d5063993d00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Oct 30 13:12:41.030547 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:41.030511 5029 patch_prober.go:29] interesting pod/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Oct 30 13:12:41.030928 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:41.030579 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" podUID="0840bdc7-256c-4b2b-a76a-8d5063993d00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Oct 30 13:12:41.102579 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:41.102548 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="06d7d761-d47f-4aa5-b931-46f910b65597" path="/var/lib/kubelet/pods/06d7d761-d47f-4aa5-b931-46f910b65597/volumes" Oct 30 13:12:41.103260 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:41.103237 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b878b27-465c-433d-aba4-e31849cfc14c" path="/var/lib/kubelet/pods/2b878b27-465c-433d-aba4-e31849cfc14c/volumes" Oct 30 13:12:41.103777 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:41.103761 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" path="/var/lib/kubelet/pods/cd68fdee-30c0-42ac-bf6e-aff5d2abb46d/volumes" Oct 30 13:12:42.029570 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:42.029524 5029 patch_prober.go:29] interesting pod/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Oct 30 13:12:42.029792 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:42.029585 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" podUID="0840bdc7-256c-4b2b-a76a-8d5063993d00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Oct 30 13:12:43.030237 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:43.030196 5029 patch_prober.go:29] interesting pod/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Oct 30 13:12:43.030658 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:43.030253 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" podUID="0840bdc7-256c-4b2b-a76a-8d5063993d00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Oct 30 13:12:44.033871 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:44.033838 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:44.034267 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:44.034166 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:44.034744 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:44.034726 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:12:44.882645 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:12:44.882608 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:12:45.106417 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:45.106384 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx"] Oct 30 13:12:51.493239 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:51.493194 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5"] Oct 30 13:12:51.494003 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:51.493950 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" podUID="0e23ddba-6739-4ccb-803a-d19cb9501b95" containerName="maas-api" containerID="cri-o://d8d99b32b06183bd0d8eeacce1973675d964d950589552ee722d8bfa30d060bf" gracePeriod=30 Oct 30 13:12:51.812768 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:51.812746 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" Oct 30 13:12:51.870452 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:51.870421 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0e23ddba-6739-4ccb-803a-d19cb9501b95-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e23ddba-6739-4ccb-803a-d19cb9501b95-maas-api-tls\") pod \"0e23ddba-6739-4ccb-803a-d19cb9501b95\" (UID: \"0e23ddba-6739-4ccb-803a-d19cb9501b95\") " Oct 30 13:12:51.870635 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:51.870516 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e23ddba-6739-4ccb-803a-d19cb9501b95-kube-api-access-5tcdq\" (UniqueName: \"kubernetes.io/projected/0e23ddba-6739-4ccb-803a-d19cb9501b95-kube-api-access-5tcdq\") pod \"0e23ddba-6739-4ccb-803a-d19cb9501b95\" (UID: \"0e23ddba-6739-4ccb-803a-d19cb9501b95\") " Oct 30 13:12:51.870700 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:51.870650 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0e23ddba-6739-4ccb-803a-d19cb9501b95-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e23ddba-6739-4ccb-803a-d19cb9501b95-metrics-certs\") pod \"0e23ddba-6739-4ccb-803a-d19cb9501b95\" (UID: \"0e23ddba-6739-4ccb-803a-d19cb9501b95\") " Oct 30 13:12:51.870755 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:51.870726 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e23ddba-6739-4ccb-803a-d19cb9501b95-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e23ddba-6739-4ccb-803a-d19cb9501b95-openshift-service-ca\") pod \"0e23ddba-6739-4ccb-803a-d19cb9501b95\" (UID: \"0e23ddba-6739-4ccb-803a-d19cb9501b95\") " Oct 30 13:12:51.871215 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:51.871167 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e23ddba-6739-4ccb-803a-d19cb9501b95-openshift-service-ca" pod "0e23ddba-6739-4ccb-803a-d19cb9501b95" (UID: "0e23ddba-6739-4ccb-803a-d19cb9501b95"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:12:51.872738 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:51.872695 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e23ddba-6739-4ccb-803a-d19cb9501b95-kube-api-access-5tcdq" pod "0e23ddba-6739-4ccb-803a-d19cb9501b95" (UID: "0e23ddba-6739-4ccb-803a-d19cb9501b95"). InnerVolumeSpecName "kube-api-access-5tcdq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:12:51.873122 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:51.873098 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e23ddba-6739-4ccb-803a-d19cb9501b95-maas-api-tls" pod "0e23ddba-6739-4ccb-803a-d19cb9501b95" (UID: "0e23ddba-6739-4ccb-803a-d19cb9501b95"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:12:51.873224 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:51.873175 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e23ddba-6739-4ccb-803a-d19cb9501b95-metrics-certs" pod "0e23ddba-6739-4ccb-803a-d19cb9501b95" (UID: "0e23ddba-6739-4ccb-803a-d19cb9501b95"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:12:51.972195 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:51.972158 5029 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e23ddba-6739-4ccb-803a-d19cb9501b95-metrics-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:51.972195 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:51.972190 5029 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e23ddba-6739-4ccb-803a-d19cb9501b95-openshift-service-ca\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:51.972195 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:51.972201 5029 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e23ddba-6739-4ccb-803a-d19cb9501b95-maas-api-tls\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:51.972546 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:51.972210 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tcdq\" (UniqueName: \"kubernetes.io/projected/0e23ddba-6739-4ccb-803a-d19cb9501b95-kube-api-access-5tcdq\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:12:52.497117 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:52.497079 5029 generic.go:356] "Generic (PLEG): container finished" podID="0e23ddba-6739-4ccb-803a-d19cb9501b95" containerID="d8d99b32b06183bd0d8eeacce1973675d964d950589552ee722d8bfa30d060bf" exitCode=0 Oct 30 13:12:52.497570 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:52.497147 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" Oct 30 13:12:52.497570 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:52.497164 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" event={"ID":"0e23ddba-6739-4ccb-803a-d19cb9501b95","Type":"ContainerDied","Data":"d8d99b32b06183bd0d8eeacce1973675d964d950589552ee722d8bfa30d060bf"} Oct 30 13:12:52.497570 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:52.497238 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5" event={"ID":"0e23ddba-6739-4ccb-803a-d19cb9501b95","Type":"ContainerDied","Data":"bac1bf406f4fe350e833c591bb59a3c702bd1fd82efa50ff21437e14c0308447"} Oct 30 13:12:52.497570 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:52.497282 5029 scope.go:122] "RemoveContainer" containerID="d8d99b32b06183bd0d8eeacce1973675d964d950589552ee722d8bfa30d060bf" Oct 30 13:12:52.507594 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:52.507578 5029 scope.go:122] "RemoveContainer" containerID="d8d99b32b06183bd0d8eeacce1973675d964d950589552ee722d8bfa30d060bf" Oct 30 13:12:52.507919 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:12:52.507877 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8d99b32b06183bd0d8eeacce1973675d964d950589552ee722d8bfa30d060bf\": container with ID starting with d8d99b32b06183bd0d8eeacce1973675d964d950589552ee722d8bfa30d060bf not found: ID does not exist" containerID="d8d99b32b06183bd0d8eeacce1973675d964d950589552ee722d8bfa30d060bf" Oct 30 13:12:52.508007 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:52.507918 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8d99b32b06183bd0d8eeacce1973675d964d950589552ee722d8bfa30d060bf"} err="failed to get container status \"d8d99b32b06183bd0d8eeacce1973675d964d950589552ee722d8bfa30d060bf\": rpc error: code = NotFound desc = could not find container \"d8d99b32b06183bd0d8eeacce1973675d964d950589552ee722d8bfa30d060bf\": container with ID starting with d8d99b32b06183bd0d8eeacce1973675d964d950589552ee722d8bfa30d060bf not found: ID does not exist" Oct 30 13:12:52.558664 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:52.558633 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5"] Oct 30 13:12:52.569675 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:52.569648 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-54994c-77b66bb4f4-jlgx5"] Oct 30 13:12:53.089185 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:53.089143 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e23ddba-6739-4ccb-803a-d19cb9501b95" path="/var/lib/kubelet/pods/0e23ddba-6739-4ccb-803a-d19cb9501b95/volumes" Oct 30 13:12:55.048088 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:55.047593 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg"] Oct 30 13:12:55.048088 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:55.047849 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" podUID="0840bdc7-256c-4b2b-a76a-8d5063993d00" containerName="istio-proxy" containerID="cri-o://aa8578681b9f810eeb6f70b9f958a91120b989554a9853c1fbe8a3626cee557b" gracePeriod=30 Oct 30 13:12:59.036011 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:59.035973 5029 patch_prober.go:29] interesting pod/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 13:12:59.036416 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:12:59.036051 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" podUID="0840bdc7-256c-4b2b-a76a-8d5063993d00" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 13:13:00.138733 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.138669 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" podUID="d6bdcd3f-0200-48e6-af85-2d1d8f022054" containerName="main" containerID="cri-o://0bd86e413d69efb0fb7226ec133d467fc74145d8b04c512bba2c836baabcca00" gracePeriod=45 Oct 30 13:13:00.413961 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.413934 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:13:00.442071 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.442049 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:13:00.487541 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.487509 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-token\" (UniqueName: \"kubernetes.io/projected/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-token\") pod \"0840bdc7-256c-4b2b-a76a-8d5063993d00\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " Oct 30 13:13:00.487713 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.487573 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-data\") pod \"0840bdc7-256c-4b2b-a76a-8d5063993d00\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " Oct 30 13:13:00.487713 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.487605 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0840bdc7-256c-4b2b-a76a-8d5063993d00-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0840bdc7-256c-4b2b-a76a-8d5063993d00-istiod-ca-cert\") pod \"0840bdc7-256c-4b2b-a76a-8d5063993d00\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " Oct 30 13:13:00.487713 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.487672 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-podinfo\") pod \"0840bdc7-256c-4b2b-a76a-8d5063993d00\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " Oct 30 13:13:00.487883 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.487722 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-workload-socket\") pod \"0840bdc7-256c-4b2b-a76a-8d5063993d00\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " Oct 30 13:13:00.487883 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.487816 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0840bdc7-256c-4b2b-a76a-8d5063993d00-kube-api-access-pqf4m\" (UniqueName: \"kubernetes.io/projected/0840bdc7-256c-4b2b-a76a-8d5063993d00-kube-api-access-pqf4m\") pod \"0840bdc7-256c-4b2b-a76a-8d5063993d00\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " Oct 30 13:13:00.487883 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.487868 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-credential-socket\") pod \"0840bdc7-256c-4b2b-a76a-8d5063993d00\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " Oct 30 13:13:00.488037 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.487952 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-workload-certs\") pod \"0840bdc7-256c-4b2b-a76a-8d5063993d00\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " Oct 30 13:13:00.488037 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.487986 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-envoy\") pod \"0840bdc7-256c-4b2b-a76a-8d5063993d00\" (UID: \"0840bdc7-256c-4b2b-a76a-8d5063993d00\") " Oct 30 13:13:00.488974 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.488943 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-workload-certs" pod "0840bdc7-256c-4b2b-a76a-8d5063993d00" (UID: "0840bdc7-256c-4b2b-a76a-8d5063993d00"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:00.488974 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.488956 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-data" pod "0840bdc7-256c-4b2b-a76a-8d5063993d00" (UID: "0840bdc7-256c-4b2b-a76a-8d5063993d00"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:00.489227 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.489200 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-credential-socket" pod "0840bdc7-256c-4b2b-a76a-8d5063993d00" (UID: "0840bdc7-256c-4b2b-a76a-8d5063993d00"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:00.489386 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.489362 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0840bdc7-256c-4b2b-a76a-8d5063993d00-istiod-ca-cert" pod "0840bdc7-256c-4b2b-a76a-8d5063993d00" (UID: "0840bdc7-256c-4b2b-a76a-8d5063993d00"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:13:00.489493 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.489426 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-workload-socket" pod "0840bdc7-256c-4b2b-a76a-8d5063993d00" (UID: "0840bdc7-256c-4b2b-a76a-8d5063993d00"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:00.490641 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.490602 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-token" pod "0840bdc7-256c-4b2b-a76a-8d5063993d00" (UID: "0840bdc7-256c-4b2b-a76a-8d5063993d00"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:00.490641 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.490621 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-envoy" pod "0840bdc7-256c-4b2b-a76a-8d5063993d00" (UID: "0840bdc7-256c-4b2b-a76a-8d5063993d00"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:00.491196 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.491156 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-podinfo" pod "0840bdc7-256c-4b2b-a76a-8d5063993d00" (UID: "0840bdc7-256c-4b2b-a76a-8d5063993d00"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 13:13:00.491543 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.491509 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0840bdc7-256c-4b2b-a76a-8d5063993d00-kube-api-access-pqf4m" pod "0840bdc7-256c-4b2b-a76a-8d5063993d00" (UID: "0840bdc7-256c-4b2b-a76a-8d5063993d00"). InnerVolumeSpecName "kube-api-access-pqf4m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:00.549814 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.549778 5029 generic.go:356] "Generic (PLEG): container finished" podID="0840bdc7-256c-4b2b-a76a-8d5063993d00" containerID="aa8578681b9f810eeb6f70b9f958a91120b989554a9853c1fbe8a3626cee557b" exitCode=0 Oct 30 13:13:00.550046 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.550022 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" Oct 30 13:13:00.550165 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.549897 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" event={"ID":"0840bdc7-256c-4b2b-a76a-8d5063993d00","Type":"ContainerDied","Data":"aa8578681b9f810eeb6f70b9f958a91120b989554a9853c1fbe8a3626cee557b"} Oct 30 13:13:00.550165 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.550134 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg" event={"ID":"0840bdc7-256c-4b2b-a76a-8d5063993d00","Type":"ContainerDied","Data":"beb3ff2c2bb791f13252a50162bebc92b49d8fc77984274d7bc6db6ec86f4027"} Oct 30 13:13:00.550262 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.550167 5029 scope.go:122] "RemoveContainer" containerID="aa8578681b9f810eeb6f70b9f958a91120b989554a9853c1fbe8a3626cee557b" Oct 30 13:13:00.552338 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.552318 5029 generic.go:356] "Generic (PLEG): container finished" podID="d6bdcd3f-0200-48e6-af85-2d1d8f022054" containerID="0bd86e413d69efb0fb7226ec133d467fc74145d8b04c512bba2c836baabcca00" exitCode=0 Oct 30 13:13:00.552493 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.552374 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" event={"ID":"d6bdcd3f-0200-48e6-af85-2d1d8f022054","Type":"ContainerDied","Data":"0bd86e413d69efb0fb7226ec133d467fc74145d8b04c512bba2c836baabcca00"} Oct 30 13:13:00.552493 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.552402 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" event={"ID":"d6bdcd3f-0200-48e6-af85-2d1d8f022054","Type":"ContainerDied","Data":"613594ad4a961182d891fb06206dd1aa7aa3c0ecfc11ffe6a7b2af31da0174d2"} Oct 30 13:13:00.552493 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.552462 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx" Oct 30 13:13:00.563282 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.562792 5029 scope.go:122] "RemoveContainer" containerID="aa8578681b9f810eeb6f70b9f958a91120b989554a9853c1fbe8a3626cee557b" Oct 30 13:13:00.563282 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:13:00.563118 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa8578681b9f810eeb6f70b9f958a91120b989554a9853c1fbe8a3626cee557b\": container with ID starting with aa8578681b9f810eeb6f70b9f958a91120b989554a9853c1fbe8a3626cee557b not found: ID does not exist" containerID="aa8578681b9f810eeb6f70b9f958a91120b989554a9853c1fbe8a3626cee557b" Oct 30 13:13:00.563282 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.563150 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa8578681b9f810eeb6f70b9f958a91120b989554a9853c1fbe8a3626cee557b"} err="failed to get container status \"aa8578681b9f810eeb6f70b9f958a91120b989554a9853c1fbe8a3626cee557b\": rpc error: code = NotFound desc = could not find container \"aa8578681b9f810eeb6f70b9f958a91120b989554a9853c1fbe8a3626cee557b\": container with ID starting with aa8578681b9f810eeb6f70b9f958a91120b989554a9853c1fbe8a3626cee557b not found: ID does not exist" Oct 30 13:13:00.563282 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.563172 5029 scope.go:122] "RemoveContainer" containerID="0bd86e413d69efb0fb7226ec133d467fc74145d8b04c512bba2c836baabcca00" Oct 30 13:13:00.574203 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.574155 5029 scope.go:122] "RemoveContainer" containerID="0bd86e413d69efb0fb7226ec133d467fc74145d8b04c512bba2c836baabcca00" Oct 30 13:13:00.574417 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:13:00.574394 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bd86e413d69efb0fb7226ec133d467fc74145d8b04c512bba2c836baabcca00\": container with ID starting with 0bd86e413d69efb0fb7226ec133d467fc74145d8b04c512bba2c836baabcca00 not found: ID does not exist" containerID="0bd86e413d69efb0fb7226ec133d467fc74145d8b04c512bba2c836baabcca00" Oct 30 13:13:00.574503 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.574434 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bd86e413d69efb0fb7226ec133d467fc74145d8b04c512bba2c836baabcca00"} err="failed to get container status \"0bd86e413d69efb0fb7226ec133d467fc74145d8b04c512bba2c836baabcca00\": rpc error: code = NotFound desc = could not find container \"0bd86e413d69efb0fb7226ec133d467fc74145d8b04c512bba2c836baabcca00\": container with ID starting with 0bd86e413d69efb0fb7226ec133d467fc74145d8b04c512bba2c836baabcca00 not found: ID does not exist" Oct 30 13:13:00.589224 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.589197 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-tmp-dir\") pod \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " Oct 30 13:13:00.589355 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.589251 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-dshm\") pod \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " Oct 30 13:13:00.589402 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.589361 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6bdcd3f-0200-48e6-af85-2d1d8f022054-kube-api-access-gsc2b\" (UniqueName: \"kubernetes.io/projected/d6bdcd3f-0200-48e6-af85-2d1d8f022054-kube-api-access-gsc2b\") pod \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " Oct 30 13:13:00.589458 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.589413 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-model-cache\") pod \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " Oct 30 13:13:00.589515 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.589457 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-home\" (UniqueName: \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-home\") pod \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " Oct 30 13:13:00.589515 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.589491 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-tmp-dir" pod "d6bdcd3f-0200-48e6-af85-2d1d8f022054" (UID: "d6bdcd3f-0200-48e6-af85-2d1d8f022054"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:00.589627 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.589533 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d6bdcd3f-0200-48e6-af85-2d1d8f022054-tls-certs\" (UniqueName: \"kubernetes.io/secret/d6bdcd3f-0200-48e6-af85-2d1d8f022054-tls-certs\") pod \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\" (UID: \"d6bdcd3f-0200-48e6-af85-2d1d8f022054\") " Oct 30 13:13:00.589817 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.589789 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-home" pod "d6bdcd3f-0200-48e6-af85-2d1d8f022054" (UID: "d6bdcd3f-0200-48e6-af85-2d1d8f022054"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:00.590165 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.590059 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-credential-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:00.590256 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.590197 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-model-cache" pod "d6bdcd3f-0200-48e6-af85-2d1d8f022054" (UID: "d6bdcd3f-0200-48e6-af85-2d1d8f022054"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:00.590364 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.590336 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-workload-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:00.590503 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.590370 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-envoy\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:00.590503 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.590386 5029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-tmp-dir\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:00.590503 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.590399 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:00.590503 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.590411 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-data\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:00.590503 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.590425 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0840bdc7-256c-4b2b-a76a-8d5063993d00-istiod-ca-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:00.590503 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.590439 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0840bdc7-256c-4b2b-a76a-8d5063993d00-istio-podinfo\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:00.590503 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.590452 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0840bdc7-256c-4b2b-a76a-8d5063993d00-workload-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:00.590503 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.590467 5029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-home\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:00.590503 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.590483 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqf4m\" (UniqueName: \"kubernetes.io/projected/0840bdc7-256c-4b2b-a76a-8d5063993d00-kube-api-access-pqf4m\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:00.591493 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.591466 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-dshm" pod "d6bdcd3f-0200-48e6-af85-2d1d8f022054" (UID: "d6bdcd3f-0200-48e6-af85-2d1d8f022054"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:00.591605 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.591582 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6bdcd3f-0200-48e6-af85-2d1d8f022054-tls-certs" pod "d6bdcd3f-0200-48e6-af85-2d1d8f022054" (UID: "d6bdcd3f-0200-48e6-af85-2d1d8f022054"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:13:00.591684 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.591668 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6bdcd3f-0200-48e6-af85-2d1d8f022054-kube-api-access-gsc2b" pod "d6bdcd3f-0200-48e6-af85-2d1d8f022054" (UID: "d6bdcd3f-0200-48e6-af85-2d1d8f022054"). InnerVolumeSpecName "kube-api-access-gsc2b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:00.631205 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.631126 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg"] Oct 30 13:13:00.635617 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.635582 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-ac0c58-openshift-default-77ff8f86f5-mkxgg"] Oct 30 13:13:00.692191 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.692160 5029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6bdcd3f-0200-48e6-af85-2d1d8f022054-tls-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:00.692191 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.692188 5029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-dshm\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:00.692191 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.692198 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gsc2b\" (UniqueName: \"kubernetes.io/projected/d6bdcd3f-0200-48e6-af85-2d1d8f022054-kube-api-access-gsc2b\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:00.692424 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.692208 5029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6bdcd3f-0200-48e6-af85-2d1d8f022054-model-cache\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:00.898497 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.898428 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx"] Oct 30 13:13:00.908475 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:00.908449 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-bb71e2/simulator-w2-bb71e2-kserve-bcc5fd544-d6hxx"] Oct 30 13:13:01.091255 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:01.091227 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0840bdc7-256c-4b2b-a76a-8d5063993d00" path="/var/lib/kubelet/pods/0840bdc7-256c-4b2b-a76a-8d5063993d00/volumes" Oct 30 13:13:01.091718 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:01.091703 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d6bdcd3f-0200-48e6-af85-2d1d8f022054" path="/var/lib/kubelet/pods/d6bdcd3f-0200-48e6-af85-2d1d8f022054/volumes" Oct 30 13:13:02.634679 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:02.634633 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks"] Oct 30 13:13:02.635149 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:02.634918 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" podUID="efa3fa51-d5f3-49c2-9658-378b1a6a2719" containerName="istio-proxy" containerID="cri-o://3d835359de3e69426b54ae7206758838d79f8a20a7fd38d7635f329a26904f71" gracePeriod=30 Oct 30 13:13:02.649214 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:02.649182 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q"] Oct 30 13:13:02.649402 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:02.649380 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" podUID="48dfe4a1-9d55-4f04-812a-1d0084f01c7e" containerName="istio-proxy" containerID="cri-o://1d7914ec89b193b3f4151530bab81d31d1928575b9b3d5e0e6dcefed7991df7b" gracePeriod=30 Oct 30 13:13:02.659492 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:02.659461 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8"] Oct 30 13:13:02.659687 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:02.659665 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" podUID="a2dc4823-8550-4c35-a226-6440fbbffbc5" containerName="istio-proxy" containerID="cri-o://9dd56de4da3541e23ca5549bbf7d8bfb78f7e52af03417554c1886d0a3a9b904" gracePeriod=30 Oct 30 13:13:06.073166 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.073133 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b"] Oct 30 13:13:06.073632 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.073342 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" podUID="60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4" containerName="maas-api" containerID="cri-o://eb71e1d2b394cdb4145fe1893c8bc352f5aa2f7c6995c92f65b662bfc0ee0bbb" gracePeriod=30 Oct 30 13:13:06.388705 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.388359 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" Oct 30 13:13:06.457986 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.457955 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-kube-api-access-4s8rr\" (UniqueName: \"kubernetes.io/projected/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-kube-api-access-4s8rr\") pod \"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4\" (UID: \"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4\") " Oct 30 13:13:06.458160 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.458040 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-openshift-service-ca\") pod \"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4\" (UID: \"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4\") " Oct 30 13:13:06.458160 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.458100 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-maas-api-tls\") pod \"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4\" (UID: \"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4\") " Oct 30 13:13:06.458296 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.458257 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-metrics-certs\") pod \"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4\" (UID: \"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4\") " Oct 30 13:13:06.458498 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.458448 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-openshift-service-ca" pod "60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4" (UID: "60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:13:06.459428 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.459400 5029 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-openshift-service-ca\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:06.460881 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.460836 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-maas-api-tls" pod "60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4" (UID: "60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:13:06.461138 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.460892 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-metrics-certs" pod "60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4" (UID: "60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:13:06.461220 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.461150 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-kube-api-access-4s8rr" pod "60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4" (UID: "60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4"). InnerVolumeSpecName "kube-api-access-4s8rr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:06.560248 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.560212 5029 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-maas-api-tls\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:06.560248 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.560246 5029 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-metrics-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:06.560452 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.560258 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4s8rr\" (UniqueName: \"kubernetes.io/projected/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4-kube-api-access-4s8rr\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:06.588746 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.588711 5029 generic.go:356] "Generic (PLEG): container finished" podID="60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4" containerID="eb71e1d2b394cdb4145fe1893c8bc352f5aa2f7c6995c92f65b662bfc0ee0bbb" exitCode=0 Oct 30 13:13:06.588916 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.588759 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" Oct 30 13:13:06.588916 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.588777 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" event={"ID":"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4","Type":"ContainerDied","Data":"eb71e1d2b394cdb4145fe1893c8bc352f5aa2f7c6995c92f65b662bfc0ee0bbb"} Oct 30 13:13:06.588916 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.588830 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b" event={"ID":"60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4","Type":"ContainerDied","Data":"890dfd7d85bb884f9413a38e90180fbb9bd57edbe7915e417874bd2f0c3a593e"} Oct 30 13:13:06.588916 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.588861 5029 scope.go:122] "RemoveContainer" containerID="eb71e1d2b394cdb4145fe1893c8bc352f5aa2f7c6995c92f65b662bfc0ee0bbb" Oct 30 13:13:06.598929 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.598908 5029 scope.go:122] "RemoveContainer" containerID="eb71e1d2b394cdb4145fe1893c8bc352f5aa2f7c6995c92f65b662bfc0ee0bbb" Oct 30 13:13:06.599197 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:13:06.599177 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb71e1d2b394cdb4145fe1893c8bc352f5aa2f7c6995c92f65b662bfc0ee0bbb\": container with ID starting with eb71e1d2b394cdb4145fe1893c8bc352f5aa2f7c6995c92f65b662bfc0ee0bbb not found: ID does not exist" containerID="eb71e1d2b394cdb4145fe1893c8bc352f5aa2f7c6995c92f65b662bfc0ee0bbb" Oct 30 13:13:06.599263 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.599204 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb71e1d2b394cdb4145fe1893c8bc352f5aa2f7c6995c92f65b662bfc0ee0bbb"} err="failed to get container status \"eb71e1d2b394cdb4145fe1893c8bc352f5aa2f7c6995c92f65b662bfc0ee0bbb\": rpc error: code = NotFound desc = could not find container \"eb71e1d2b394cdb4145fe1893c8bc352f5aa2f7c6995c92f65b662bfc0ee0bbb\": container with ID starting with eb71e1d2b394cdb4145fe1893c8bc352f5aa2f7c6995c92f65b662bfc0ee0bbb not found: ID does not exist" Oct 30 13:13:06.646006 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.645941 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b"] Oct 30 13:13:06.651691 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:06.651669 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-bb71e2-54bc8bcb69-7tz6b"] Oct 30 13:13:07.089666 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:07.089634 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4" path="/var/lib/kubelet/pods/60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4/volumes" Oct 30 13:13:07.953996 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:07.953965 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:13:07.980894 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:07.980872 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:13:07.984987 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:07.984970 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:13:08.076869 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.076826 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istiod-ca-cert\") pod \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " Oct 30 13:13:08.077015 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.076889 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-data\") pod \"a2dc4823-8550-4c35-a226-6440fbbffbc5\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " Oct 30 13:13:08.077015 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.076918 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-token\" (UniqueName: \"kubernetes.io/projected/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-token\") pod \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " Oct 30 13:13:08.077015 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.076963 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-credential-socket\") pod \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " Oct 30 13:13:08.077015 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077003 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istiod-ca-cert\") pod \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " Oct 30 13:13:08.077228 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077028 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-data\") pod \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " Oct 30 13:13:08.077228 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077057 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-token\" (UniqueName: \"kubernetes.io/projected/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-token\") pod \"a2dc4823-8550-4c35-a226-6440fbbffbc5\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " Oct 30 13:13:08.077228 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077173 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-kube-api-access-78gsn\" (UniqueName: \"kubernetes.io/projected/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-kube-api-access-78gsn\") pod \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " Oct 30 13:13:08.077228 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077208 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-workload-certs\") pod \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " Oct 30 13:13:08.077496 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077243 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a2dc4823-8550-4c35-a226-6440fbbffbc5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2dc4823-8550-4c35-a226-6440fbbffbc5-istiod-ca-cert\") pod \"a2dc4823-8550-4c35-a226-6440fbbffbc5\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " Oct 30 13:13:08.077496 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077281 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2dc4823-8550-4c35-a226-6440fbbffbc5-kube-api-access-h4kvp\" (UniqueName: \"kubernetes.io/projected/a2dc4823-8550-4c35-a226-6440fbbffbc5-kube-api-access-h4kvp\") pod \"a2dc4823-8550-4c35-a226-6440fbbffbc5\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " Oct 30 13:13:08.077496 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077344 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-token\" (UniqueName: \"kubernetes.io/projected/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-token\") pod \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " Oct 30 13:13:08.077496 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077358 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-credential-socket" pod "efa3fa51-d5f3-49c2-9658-378b1a6a2719" (UID: "efa3fa51-d5f3-49c2-9658-378b1a6a2719"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:08.077496 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077383 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-credential-socket\") pod \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " Oct 30 13:13:08.077496 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077414 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-workload-certs\") pod \"a2dc4823-8550-4c35-a226-6440fbbffbc5\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " Oct 30 13:13:08.077496 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077446 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-envoy\") pod \"a2dc4823-8550-4c35-a226-6440fbbffbc5\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " Oct 30 13:13:08.077496 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077450 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-data" pod "a2dc4823-8550-4c35-a226-6440fbbffbc5" (UID: "a2dc4823-8550-4c35-a226-6440fbbffbc5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:08.077496 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077490 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-workload-socket\") pod \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " Oct 30 13:13:08.077938 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077523 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-workload-socket\") pod \"a2dc4823-8550-4c35-a226-6440fbbffbc5\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " Oct 30 13:13:08.077938 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077559 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-envoy\") pod \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " Oct 30 13:13:08.077938 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077594 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-data\") pod \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " Oct 30 13:13:08.077938 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077623 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-credential-socket\") pod \"a2dc4823-8550-4c35-a226-6440fbbffbc5\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " Oct 30 13:13:08.077938 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077681 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-workload-socket\") pod \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " Oct 30 13:13:08.077938 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077714 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-podinfo\") pod \"a2dc4823-8550-4c35-a226-6440fbbffbc5\" (UID: \"a2dc4823-8550-4c35-a226-6440fbbffbc5\") " Oct 30 13:13:08.077938 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077755 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-podinfo\") pod \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " Oct 30 13:13:08.077938 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077789 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-workload-certs\") pod \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\" (UID: \"48dfe4a1-9d55-4f04-812a-1d0084f01c7e\") " Oct 30 13:13:08.077938 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077848 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-podinfo\") pod \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " Oct 30 13:13:08.077938 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077909 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/efa3fa51-d5f3-49c2-9658-378b1a6a2719-kube-api-access-5mzsf\" (UniqueName: \"kubernetes.io/projected/efa3fa51-d5f3-49c2-9658-378b1a6a2719-kube-api-access-5mzsf\") pod \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " Oct 30 13:13:08.078483 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077953 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-envoy\") pod \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\" (UID: \"efa3fa51-d5f3-49c2-9658-378b1a6a2719\") " Oct 30 13:13:08.078798 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.078628 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-data\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.078798 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.078656 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-credential-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.078798 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.077523 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istiod-ca-cert" pod "efa3fa51-d5f3-49c2-9658-378b1a6a2719" (UID: "efa3fa51-d5f3-49c2-9658-378b1a6a2719"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:13:08.078984 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.078040 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-workload-certs" pod "48dfe4a1-9d55-4f04-812a-1d0084f01c7e" (UID: "48dfe4a1-9d55-4f04-812a-1d0084f01c7e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:08.078984 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.078393 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istiod-ca-cert" pod "48dfe4a1-9d55-4f04-812a-1d0084f01c7e" (UID: "48dfe4a1-9d55-4f04-812a-1d0084f01c7e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:13:08.078984 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.078644 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-workload-socket" pod "48dfe4a1-9d55-4f04-812a-1d0084f01c7e" (UID: "48dfe4a1-9d55-4f04-812a-1d0084f01c7e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:08.078984 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.078762 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-workload-socket" pod "efa3fa51-d5f3-49c2-9658-378b1a6a2719" (UID: "efa3fa51-d5f3-49c2-9658-378b1a6a2719"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:08.079279 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.079119 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-credential-socket" pod "a2dc4823-8550-4c35-a226-6440fbbffbc5" (UID: "a2dc4823-8550-4c35-a226-6440fbbffbc5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:08.079549 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.079527 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-data" pod "efa3fa51-d5f3-49c2-9658-378b1a6a2719" (UID: "efa3fa51-d5f3-49c2-9658-378b1a6a2719"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:08.079715 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.079684 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2dc4823-8550-4c35-a226-6440fbbffbc5-istiod-ca-cert" pod "a2dc4823-8550-4c35-a226-6440fbbffbc5" (UID: "a2dc4823-8550-4c35-a226-6440fbbffbc5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:13:08.079927 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.079900 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-data" pod "48dfe4a1-9d55-4f04-812a-1d0084f01c7e" (UID: "48dfe4a1-9d55-4f04-812a-1d0084f01c7e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:08.080009 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.079924 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-workload-socket" pod "a2dc4823-8550-4c35-a226-6440fbbffbc5" (UID: "a2dc4823-8550-4c35-a226-6440fbbffbc5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:08.080347 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.080324 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-workload-certs" pod "efa3fa51-d5f3-49c2-9658-378b1a6a2719" (UID: "efa3fa51-d5f3-49c2-9658-378b1a6a2719"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:08.080451 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.080382 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-credential-socket" pod "48dfe4a1-9d55-4f04-812a-1d0084f01c7e" (UID: "48dfe4a1-9d55-4f04-812a-1d0084f01c7e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:08.080793 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.080769 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-workload-certs" pod "a2dc4823-8550-4c35-a226-6440fbbffbc5" (UID: "a2dc4823-8550-4c35-a226-6440fbbffbc5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:08.085189 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.085147 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-podinfo" pod "48dfe4a1-9d55-4f04-812a-1d0084f01c7e" (UID: "48dfe4a1-9d55-4f04-812a-1d0084f01c7e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 13:13:08.085288 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.085249 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-envoy" pod "48dfe4a1-9d55-4f04-812a-1d0084f01c7e" (UID: "48dfe4a1-9d55-4f04-812a-1d0084f01c7e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:08.085399 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.085377 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-token" pod "efa3fa51-d5f3-49c2-9658-378b1a6a2719" (UID: "efa3fa51-d5f3-49c2-9658-378b1a6a2719"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:08.085841 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.085577 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-envoy" pod "efa3fa51-d5f3-49c2-9658-378b1a6a2719" (UID: "efa3fa51-d5f3-49c2-9658-378b1a6a2719"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:08.085972 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.085669 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2dc4823-8550-4c35-a226-6440fbbffbc5-kube-api-access-h4kvp" pod "a2dc4823-8550-4c35-a226-6440fbbffbc5" (UID: "a2dc4823-8550-4c35-a226-6440fbbffbc5"). InnerVolumeSpecName "kube-api-access-h4kvp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:08.085972 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.085711 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-podinfo" pod "efa3fa51-d5f3-49c2-9658-378b1a6a2719" (UID: "efa3fa51-d5f3-49c2-9658-378b1a6a2719"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 13:13:08.086161 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.085755 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efa3fa51-d5f3-49c2-9658-378b1a6a2719-kube-api-access-5mzsf" pod "efa3fa51-d5f3-49c2-9658-378b1a6a2719" (UID: "efa3fa51-d5f3-49c2-9658-378b1a6a2719"). InnerVolumeSpecName "kube-api-access-5mzsf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:08.086161 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.085795 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-podinfo" pod "a2dc4823-8550-4c35-a226-6440fbbffbc5" (UID: "a2dc4823-8550-4c35-a226-6440fbbffbc5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 13:13:08.086161 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.085407 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-envoy" pod "a2dc4823-8550-4c35-a226-6440fbbffbc5" (UID: "a2dc4823-8550-4c35-a226-6440fbbffbc5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:08.086438 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.086414 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-kube-api-access-78gsn" pod "48dfe4a1-9d55-4f04-812a-1d0084f01c7e" (UID: "48dfe4a1-9d55-4f04-812a-1d0084f01c7e"). InnerVolumeSpecName "kube-api-access-78gsn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:08.086629 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.086614 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-token" pod "48dfe4a1-9d55-4f04-812a-1d0084f01c7e" (UID: "48dfe4a1-9d55-4f04-812a-1d0084f01c7e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:08.086761 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.086743 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-token" pod "a2dc4823-8550-4c35-a226-6440fbbffbc5" (UID: "a2dc4823-8550-4c35-a226-6440fbbffbc5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:08.180564 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180485 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.180564 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180523 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istiod-ca-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.180564 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180536 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-data\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.180564 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180547 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.180564 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180560 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78gsn\" (UniqueName: \"kubernetes.io/projected/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-kube-api-access-78gsn\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.181200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180573 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-workload-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.181200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180586 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2dc4823-8550-4c35-a226-6440fbbffbc5-istiod-ca-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.181200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180598 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4kvp\" (UniqueName: \"kubernetes.io/projected/a2dc4823-8550-4c35-a226-6440fbbffbc5-kube-api-access-h4kvp\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.181200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180611 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.181200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180623 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-credential-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.181200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180634 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-workload-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.181200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180646 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-envoy\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.181200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180660 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-workload-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.181200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180674 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-workload-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.181200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180687 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-envoy\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.181200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180698 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-data\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.181200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180709 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2dc4823-8550-4c35-a226-6440fbbffbc5-credential-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.181200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180721 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-workload-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.181200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180733 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2dc4823-8550-4c35-a226-6440fbbffbc5-istio-podinfo\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.181200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180744 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-istio-podinfo\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.181200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180757 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48dfe4a1-9d55-4f04-812a-1d0084f01c7e-workload-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.181200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180769 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-podinfo\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.181200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180781 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mzsf\" (UniqueName: \"kubernetes.io/projected/efa3fa51-d5f3-49c2-9658-378b1a6a2719-kube-api-access-5mzsf\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.181200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180795 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istio-envoy\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.181200 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.180807 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efa3fa51-d5f3-49c2-9658-378b1a6a2719-istiod-ca-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:08.601495 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.601450 5029 generic.go:356] "Generic (PLEG): container finished" podID="a2dc4823-8550-4c35-a226-6440fbbffbc5" containerID="9dd56de4da3541e23ca5549bbf7d8bfb78f7e52af03417554c1886d0a3a9b904" exitCode=0 Oct 30 13:13:08.601495 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.601496 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" Oct 30 13:13:08.601748 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.601524 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" event={"ID":"a2dc4823-8550-4c35-a226-6440fbbffbc5","Type":"ContainerDied","Data":"9dd56de4da3541e23ca5549bbf7d8bfb78f7e52af03417554c1886d0a3a9b904"} Oct 30 13:13:08.601748 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.601581 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8" event={"ID":"a2dc4823-8550-4c35-a226-6440fbbffbc5","Type":"ContainerDied","Data":"ce073ed4d6089ea28daef2fda4dbfbb3476f63c88dea7df5cfbc57715ca4dca1"} Oct 30 13:13:08.601748 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.601611 5029 scope.go:122] "RemoveContainer" containerID="9dd56de4da3541e23ca5549bbf7d8bfb78f7e52af03417554c1886d0a3a9b904" Oct 30 13:13:08.603264 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.603144 5029 generic.go:356] "Generic (PLEG): container finished" podID="efa3fa51-d5f3-49c2-9658-378b1a6a2719" containerID="3d835359de3e69426b54ae7206758838d79f8a20a7fd38d7635f329a26904f71" exitCode=0 Oct 30 13:13:08.603264 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.603208 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" Oct 30 13:13:08.603264 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.603248 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" event={"ID":"efa3fa51-d5f3-49c2-9658-378b1a6a2719","Type":"ContainerDied","Data":"3d835359de3e69426b54ae7206758838d79f8a20a7fd38d7635f329a26904f71"} Oct 30 13:13:08.603497 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.603303 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks" event={"ID":"efa3fa51-d5f3-49c2-9658-378b1a6a2719","Type":"ContainerDied","Data":"834ccef10eed644f050f3d94d0542555d88058527d4f9b5c359687c41f1a1882"} Oct 30 13:13:08.604936 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.604902 5029 generic.go:356] "Generic (PLEG): container finished" podID="48dfe4a1-9d55-4f04-812a-1d0084f01c7e" containerID="1d7914ec89b193b3f4151530bab81d31d1928575b9b3d5e0e6dcefed7991df7b" exitCode=0 Oct 30 13:13:08.605047 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.604931 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" event={"ID":"48dfe4a1-9d55-4f04-812a-1d0084f01c7e","Type":"ContainerDied","Data":"1d7914ec89b193b3f4151530bab81d31d1928575b9b3d5e0e6dcefed7991df7b"} Oct 30 13:13:08.605047 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.604963 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" Oct 30 13:13:08.605047 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.604990 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q" event={"ID":"48dfe4a1-9d55-4f04-812a-1d0084f01c7e","Type":"ContainerDied","Data":"cd867d6c68659bac758b53b05172a7a1b63de19251c4c628866946f133735c89"} Oct 30 13:13:08.613598 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.613521 5029 scope.go:122] "RemoveContainer" containerID="9dd56de4da3541e23ca5549bbf7d8bfb78f7e52af03417554c1886d0a3a9b904" Oct 30 13:13:08.613899 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:13:08.613874 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dd56de4da3541e23ca5549bbf7d8bfb78f7e52af03417554c1886d0a3a9b904\": container with ID starting with 9dd56de4da3541e23ca5549bbf7d8bfb78f7e52af03417554c1886d0a3a9b904 not found: ID does not exist" containerID="9dd56de4da3541e23ca5549bbf7d8bfb78f7e52af03417554c1886d0a3a9b904" Oct 30 13:13:08.613993 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.613906 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dd56de4da3541e23ca5549bbf7d8bfb78f7e52af03417554c1886d0a3a9b904"} err="failed to get container status \"9dd56de4da3541e23ca5549bbf7d8bfb78f7e52af03417554c1886d0a3a9b904\": rpc error: code = NotFound desc = could not find container \"9dd56de4da3541e23ca5549bbf7d8bfb78f7e52af03417554c1886d0a3a9b904\": container with ID starting with 9dd56de4da3541e23ca5549bbf7d8bfb78f7e52af03417554c1886d0a3a9b904 not found: ID does not exist" Oct 30 13:13:08.613993 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.613935 5029 scope.go:122] "RemoveContainer" containerID="3d835359de3e69426b54ae7206758838d79f8a20a7fd38d7635f329a26904f71" Oct 30 13:13:08.622410 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.622395 5029 scope.go:122] "RemoveContainer" containerID="3d835359de3e69426b54ae7206758838d79f8a20a7fd38d7635f329a26904f71" Oct 30 13:13:08.622655 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:13:08.622636 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d835359de3e69426b54ae7206758838d79f8a20a7fd38d7635f329a26904f71\": container with ID starting with 3d835359de3e69426b54ae7206758838d79f8a20a7fd38d7635f329a26904f71 not found: ID does not exist" containerID="3d835359de3e69426b54ae7206758838d79f8a20a7fd38d7635f329a26904f71" Oct 30 13:13:08.622716 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.622660 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d835359de3e69426b54ae7206758838d79f8a20a7fd38d7635f329a26904f71"} err="failed to get container status \"3d835359de3e69426b54ae7206758838d79f8a20a7fd38d7635f329a26904f71\": rpc error: code = NotFound desc = could not find container \"3d835359de3e69426b54ae7206758838d79f8a20a7fd38d7635f329a26904f71\": container with ID starting with 3d835359de3e69426b54ae7206758838d79f8a20a7fd38d7635f329a26904f71 not found: ID does not exist" Oct 30 13:13:08.622716 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.622674 5029 scope.go:122] "RemoveContainer" containerID="1d7914ec89b193b3f4151530bab81d31d1928575b9b3d5e0e6dcefed7991df7b" Oct 30 13:13:08.630266 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.630249 5029 scope.go:122] "RemoveContainer" containerID="1d7914ec89b193b3f4151530bab81d31d1928575b9b3d5e0e6dcefed7991df7b" Oct 30 13:13:08.630501 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:13:08.630482 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d7914ec89b193b3f4151530bab81d31d1928575b9b3d5e0e6dcefed7991df7b\": container with ID starting with 1d7914ec89b193b3f4151530bab81d31d1928575b9b3d5e0e6dcefed7991df7b not found: ID does not exist" containerID="1d7914ec89b193b3f4151530bab81d31d1928575b9b3d5e0e6dcefed7991df7b" Oct 30 13:13:08.630545 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.630507 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d7914ec89b193b3f4151530bab81d31d1928575b9b3d5e0e6dcefed7991df7b"} err="failed to get container status \"1d7914ec89b193b3f4151530bab81d31d1928575b9b3d5e0e6dcefed7991df7b\": rpc error: code = NotFound desc = could not find container \"1d7914ec89b193b3f4151530bab81d31d1928575b9b3d5e0e6dcefed7991df7b\": container with ID starting with 1d7914ec89b193b3f4151530bab81d31d1928575b9b3d5e0e6dcefed7991df7b not found: ID does not exist" Oct 30 13:13:08.663536 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.663517 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks"] Oct 30 13:13:08.674683 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.674658 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-562ks"] Oct 30 13:13:08.710335 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.710308 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8"] Oct 30 13:13:08.716411 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.716387 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-v5mc8"] Oct 30 13:13:08.752048 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.752025 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q"] Oct 30 13:13:08.755546 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:08.755526 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-54994c-openshift-default-c8547749c-khh9q"] Oct 30 13:13:09.088273 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:09.088232 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="48dfe4a1-9d55-4f04-812a-1d0084f01c7e" path="/var/lib/kubelet/pods/48dfe4a1-9d55-4f04-812a-1d0084f01c7e/volumes" Oct 30 13:13:09.088878 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:09.088858 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a2dc4823-8550-4c35-a226-6440fbbffbc5" path="/var/lib/kubelet/pods/a2dc4823-8550-4c35-a226-6440fbbffbc5/volumes" Oct 30 13:13:09.089417 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:09.089395 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="efa3fa51-d5f3-49c2-9658-378b1a6a2719" path="/var/lib/kubelet/pods/efa3fa51-d5f3-49c2-9658-378b1a6a2719/volumes" Oct 30 13:13:14.878913 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:13:14.878878 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:13:15.721606 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:15.721574 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg"] Oct 30 13:13:15.748496 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:15.748464 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p"] Oct 30 13:13:16.388013 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:16.387975 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb"] Oct 30 13:13:16.388510 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:16.388266 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" podUID="eb32630c-51c4-495d-8a43-eed34246ada8" containerName="istio-proxy" containerID="cri-o://c142c44181093d993c3ab7115ed95392d6a049e2e7a2c9f2bbb18a7bfd434e31" gracePeriod=30 Oct 30 13:13:16.403855 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:16.403818 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9"] Oct 30 13:13:16.404105 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:16.404044 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" podUID="c461dc10-21d9-40ea-ad0b-1125e530b240" containerName="istio-proxy" containerID="cri-o://b3c0da523bd1cd996a813f7fc382f288183417a243dfd0717ce49827050b653c" gracePeriod=30 Oct 30 13:13:16.413494 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:16.412731 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn"] Oct 30 13:13:16.413494 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:16.412935 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" podUID="17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2" containerName="istio-proxy" containerID="cri-o://353b8134f8946b557d65acf9cec18d086032d1424a8fdbd1a8a0c8c7ce2a3cf3" gracePeriod=30 Oct 30 13:13:21.679563 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.679527 5029 generic.go:356] "Generic (PLEG): container finished" podID="17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2" containerID="353b8134f8946b557d65acf9cec18d086032d1424a8fdbd1a8a0c8c7ce2a3cf3" exitCode=0 Oct 30 13:13:21.679978 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.679716 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" event={"ID":"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2","Type":"ContainerDied","Data":"353b8134f8946b557d65acf9cec18d086032d1424a8fdbd1a8a0c8c7ce2a3cf3"} Oct 30 13:13:21.681554 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.681526 5029 generic.go:356] "Generic (PLEG): container finished" podID="eb32630c-51c4-495d-8a43-eed34246ada8" containerID="c142c44181093d993c3ab7115ed95392d6a049e2e7a2c9f2bbb18a7bfd434e31" exitCode=0 Oct 30 13:13:21.681669 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.681618 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" event={"ID":"eb32630c-51c4-495d-8a43-eed34246ada8","Type":"ContainerDied","Data":"c142c44181093d993c3ab7115ed95392d6a049e2e7a2c9f2bbb18a7bfd434e31"} Oct 30 13:13:21.681669 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.681663 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" event={"ID":"eb32630c-51c4-495d-8a43-eed34246ada8","Type":"ContainerDied","Data":"325648bded87fabfadead5f4ad8b67e1b415bd905d09dafe139c6e7f3c3cdeb0"} Oct 30 13:13:21.681793 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.681690 5029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="325648bded87fabfadead5f4ad8b67e1b415bd905d09dafe139c6e7f3c3cdeb0" Oct 30 13:13:21.684041 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.684008 5029 generic.go:356] "Generic (PLEG): container finished" podID="c461dc10-21d9-40ea-ad0b-1125e530b240" containerID="b3c0da523bd1cd996a813f7fc382f288183417a243dfd0717ce49827050b653c" exitCode=0 Oct 30 13:13:21.684146 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.684114 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" event={"ID":"c461dc10-21d9-40ea-ad0b-1125e530b240","Type":"ContainerDied","Data":"b3c0da523bd1cd996a813f7fc382f288183417a243dfd0717ce49827050b653c"} Oct 30 13:13:21.691889 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.691217 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:13:21.764813 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.764149 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:13:21.770188 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.770160 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:13:21.828273 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.828239 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eb32630c-51c4-495d-8a43-eed34246ada8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb32630c-51c4-495d-8a43-eed34246ada8-istio-podinfo\") pod \"eb32630c-51c4-495d-8a43-eed34246ada8\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " Oct 30 13:13:21.828484 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.828303 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-credential-socket\") pod \"eb32630c-51c4-495d-8a43-eed34246ada8\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " Oct 30 13:13:21.828484 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.828429 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb32630c-51c4-495d-8a43-eed34246ada8-kube-api-access-jjrnx\" (UniqueName: \"kubernetes.io/projected/eb32630c-51c4-495d-8a43-eed34246ada8-kube-api-access-jjrnx\") pod \"eb32630c-51c4-495d-8a43-eed34246ada8\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " Oct 30 13:13:21.828484 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.828465 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb32630c-51c4-495d-8a43-eed34246ada8-istio-token\" (UniqueName: \"kubernetes.io/projected/eb32630c-51c4-495d-8a43-eed34246ada8-istio-token\") pod \"eb32630c-51c4-495d-8a43-eed34246ada8\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " Oct 30 13:13:21.828606 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.828501 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-istio-data\") pod \"eb32630c-51c4-495d-8a43-eed34246ada8\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " Oct 30 13:13:21.828606 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.828535 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb32630c-51c4-495d-8a43-eed34246ada8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb32630c-51c4-495d-8a43-eed34246ada8-istiod-ca-cert\") pod \"eb32630c-51c4-495d-8a43-eed34246ada8\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " Oct 30 13:13:21.828606 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.828575 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-istio-envoy\") pod \"eb32630c-51c4-495d-8a43-eed34246ada8\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " Oct 30 13:13:21.828606 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.828601 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-workload-certs\") pod \"eb32630c-51c4-495d-8a43-eed34246ada8\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " Oct 30 13:13:21.828728 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.828631 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-workload-socket\") pod \"eb32630c-51c4-495d-8a43-eed34246ada8\" (UID: \"eb32630c-51c4-495d-8a43-eed34246ada8\") " Oct 30 13:13:21.828728 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.828625 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-credential-socket" pod "eb32630c-51c4-495d-8a43-eed34246ada8" (UID: "eb32630c-51c4-495d-8a43-eed34246ada8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:21.828875 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.828854 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-istio-data" pod "eb32630c-51c4-495d-8a43-eed34246ada8" (UID: "eb32630c-51c4-495d-8a43-eed34246ada8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:21.828924 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.828911 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-credential-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:21.829253 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.829232 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb32630c-51c4-495d-8a43-eed34246ada8-istiod-ca-cert" pod "eb32630c-51c4-495d-8a43-eed34246ada8" (UID: "eb32630c-51c4-495d-8a43-eed34246ada8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:13:21.831712 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.829510 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-workload-socket" pod "eb32630c-51c4-495d-8a43-eed34246ada8" (UID: "eb32630c-51c4-495d-8a43-eed34246ada8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:21.831712 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.829569 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-workload-certs" pod "eb32630c-51c4-495d-8a43-eed34246ada8" (UID: "eb32630c-51c4-495d-8a43-eed34246ada8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:21.831712 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.831517 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb32630c-51c4-495d-8a43-eed34246ada8-istio-token" pod "eb32630c-51c4-495d-8a43-eed34246ada8" (UID: "eb32630c-51c4-495d-8a43-eed34246ada8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:21.831712 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.831592 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb32630c-51c4-495d-8a43-eed34246ada8-kube-api-access-jjrnx" pod "eb32630c-51c4-495d-8a43-eed34246ada8" (UID: "eb32630c-51c4-495d-8a43-eed34246ada8"). InnerVolumeSpecName "kube-api-access-jjrnx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:21.832429 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.832406 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-istio-envoy" pod "eb32630c-51c4-495d-8a43-eed34246ada8" (UID: "eb32630c-51c4-495d-8a43-eed34246ada8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:21.833775 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.833750 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb32630c-51c4-495d-8a43-eed34246ada8-istio-podinfo" pod "eb32630c-51c4-495d-8a43-eed34246ada8" (UID: "eb32630c-51c4-495d-8a43-eed34246ada8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 13:13:21.930273 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.930195 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-podinfo\") pod \"c461dc10-21d9-40ea-ad0b-1125e530b240\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " Oct 30 13:13:21.930273 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.930259 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istiod-ca-cert\") pod \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " Oct 30 13:13:21.930496 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.930293 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-token\" (UniqueName: \"kubernetes.io/projected/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-token\") pod \"c461dc10-21d9-40ea-ad0b-1125e530b240\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " Oct 30 13:13:21.930496 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.930338 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c461dc10-21d9-40ea-ad0b-1125e530b240-kube-api-access-gh8vx\" (UniqueName: \"kubernetes.io/projected/c461dc10-21d9-40ea-ad0b-1125e530b240-kube-api-access-gh8vx\") pod \"c461dc10-21d9-40ea-ad0b-1125e530b240\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " Oct 30 13:13:21.930496 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.930367 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c461dc10-21d9-40ea-ad0b-1125e530b240-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c461dc10-21d9-40ea-ad0b-1125e530b240-istiod-ca-cert\") pod \"c461dc10-21d9-40ea-ad0b-1125e530b240\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " Oct 30 13:13:21.930496 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.930408 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-data\") pod \"c461dc10-21d9-40ea-ad0b-1125e530b240\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " Oct 30 13:13:21.930496 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.930445 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-token\" (UniqueName: \"kubernetes.io/projected/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-token\") pod \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " Oct 30 13:13:21.930767 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.930499 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-envoy\") pod \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " Oct 30 13:13:21.930767 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.930550 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-workload-socket\") pod \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " Oct 30 13:13:21.930767 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.930600 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-data\") pod \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " Oct 30 13:13:21.930767 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.930638 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-credential-socket\") pod \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " Oct 30 13:13:21.930767 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.930669 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istiod-ca-cert" pod "17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2" (UID: "17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:13:21.930767 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.930696 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-workload-certs\") pod \"c461dc10-21d9-40ea-ad0b-1125e530b240\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " Oct 30 13:13:21.930767 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.930747 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-envoy\") pod \"c461dc10-21d9-40ea-ad0b-1125e530b240\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " Oct 30 13:13:21.931151 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.930775 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-kube-api-access-rzntb\" (UniqueName: \"kubernetes.io/projected/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-kube-api-access-rzntb\") pod \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " Oct 30 13:13:21.931151 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.930802 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-workload-socket\") pod \"c461dc10-21d9-40ea-ad0b-1125e530b240\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " Oct 30 13:13:21.931151 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.930860 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-credential-socket\") pod \"c461dc10-21d9-40ea-ad0b-1125e530b240\" (UID: \"c461dc10-21d9-40ea-ad0b-1125e530b240\") " Oct 30 13:13:21.931151 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.930895 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-podinfo\") pod \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " Oct 30 13:13:21.931151 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.930925 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-workload-certs\") pod \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\" (UID: \"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2\") " Oct 30 13:13:21.931151 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.930940 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-workload-socket" pod "17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2" (UID: "17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:21.931151 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.930962 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-data" pod "c461dc10-21d9-40ea-ad0b-1125e530b240" (UID: "c461dc10-21d9-40ea-ad0b-1125e530b240"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:21.931527 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.931369 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c461dc10-21d9-40ea-ad0b-1125e530b240-istiod-ca-cert" pod "c461dc10-21d9-40ea-ad0b-1125e530b240" (UID: "c461dc10-21d9-40ea-ad0b-1125e530b240"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:13:21.931527 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.931399 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-data\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:21.931527 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.931424 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjrnx\" (UniqueName: \"kubernetes.io/projected/eb32630c-51c4-495d-8a43-eed34246ada8-kube-api-access-jjrnx\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:21.931527 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.931441 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb32630c-51c4-495d-8a43-eed34246ada8-istio-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:21.931527 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.931456 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-istio-data\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:21.931527 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.931470 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb32630c-51c4-495d-8a43-eed34246ada8-istiod-ca-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:21.931527 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.931485 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-istio-envoy\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:21.931527 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.931499 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-workload-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:21.931527 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.931511 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-workload-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:21.931527 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.931523 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb32630c-51c4-495d-8a43-eed34246ada8-workload-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:21.932024 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.931536 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb32630c-51c4-495d-8a43-eed34246ada8-istio-podinfo\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:21.932024 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.931549 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istiod-ca-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:21.932024 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.931633 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-workload-socket" pod "c461dc10-21d9-40ea-ad0b-1125e530b240" (UID: "c461dc10-21d9-40ea-ad0b-1125e530b240"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:21.932024 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.931753 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-workload-certs" pod "17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2" (UID: "17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:21.932024 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.931992 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-credential-socket" pod "c461dc10-21d9-40ea-ad0b-1125e530b240" (UID: "c461dc10-21d9-40ea-ad0b-1125e530b240"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:21.933061 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.933028 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c461dc10-21d9-40ea-ad0b-1125e530b240-kube-api-access-gh8vx" pod "c461dc10-21d9-40ea-ad0b-1125e530b240" (UID: "c461dc10-21d9-40ea-ad0b-1125e530b240"). InnerVolumeSpecName "kube-api-access-gh8vx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:21.933268 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.933243 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-podinfo" pod "c461dc10-21d9-40ea-ad0b-1125e530b240" (UID: "c461dc10-21d9-40ea-ad0b-1125e530b240"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 13:13:21.933348 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.933270 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-credential-socket" pod "17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2" (UID: "17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:21.933488 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.933466 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-data" pod "17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2" (UID: "17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:21.933555 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.933481 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-workload-certs" pod "c461dc10-21d9-40ea-ad0b-1125e530b240" (UID: "c461dc10-21d9-40ea-ad0b-1125e530b240"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:21.933854 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.933831 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-envoy" pod "c461dc10-21d9-40ea-ad0b-1125e530b240" (UID: "c461dc10-21d9-40ea-ad0b-1125e530b240"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:21.934014 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.933991 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-token" pod "c461dc10-21d9-40ea-ad0b-1125e530b240" (UID: "c461dc10-21d9-40ea-ad0b-1125e530b240"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:21.934110 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.934030 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-kube-api-access-rzntb" pod "17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2" (UID: "17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2"). InnerVolumeSpecName "kube-api-access-rzntb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:21.934169 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.934138 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-envoy" pod "17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2" (UID: "17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:21.934251 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.934236 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-podinfo" pod "17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2" (UID: "17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 13:13:21.934675 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:21.934659 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-token" pod "17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2" (UID: "17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:22.032477 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.032403 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-credential-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:22.032477 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.032431 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-workload-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:22.032477 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.032440 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-envoy\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:22.032477 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.032449 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzntb\" (UniqueName: \"kubernetes.io/projected/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-kube-api-access-rzntb\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:22.032477 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.032460 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-workload-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:22.032477 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.032469 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c461dc10-21d9-40ea-ad0b-1125e530b240-credential-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:22.032477 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.032479 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-podinfo\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:22.032810 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.032487 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-workload-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:22.032810 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.032495 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-podinfo\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:22.032810 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.032505 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c461dc10-21d9-40ea-ad0b-1125e530b240-istio-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:22.032810 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.032514 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gh8vx\" (UniqueName: \"kubernetes.io/projected/c461dc10-21d9-40ea-ad0b-1125e530b240-kube-api-access-gh8vx\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:22.032810 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.032521 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c461dc10-21d9-40ea-ad0b-1125e530b240-istiod-ca-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:22.032810 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.032530 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:22.032810 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.032537 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-envoy\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:22.032810 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.032546 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2-istio-data\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:22.691386 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.691300 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" Oct 30 13:13:22.691386 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.691310 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9" event={"ID":"c461dc10-21d9-40ea-ad0b-1125e530b240","Type":"ContainerDied","Data":"bbded2b80d08790493d70f78419a43166dfc62c6ad72eda899ef6cf45f2cbc14"} Oct 30 13:13:22.691896 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.691389 5029 scope.go:122] "RemoveContainer" containerID="b3c0da523bd1cd996a813f7fc382f288183417a243dfd0717ce49827050b653c" Oct 30 13:13:22.692953 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.692930 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" Oct 30 13:13:22.693095 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.692932 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn" event={"ID":"17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2","Type":"ContainerDied","Data":"713c6258000cdae307c1eaccbfc33a8fa7bc604fcf84de0c5a17674fcff4be84"} Oct 30 13:13:22.693095 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.692942 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb" Oct 30 13:13:22.702328 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.702306 5029 scope.go:122] "RemoveContainer" containerID="353b8134f8946b557d65acf9cec18d086032d1424a8fdbd1a8a0c8c7ce2a3cf3" Oct 30 13:13:22.755403 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.755379 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9"] Oct 30 13:13:22.766378 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.766352 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-bbtd9"] Oct 30 13:13:22.796816 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.796774 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb"] Oct 30 13:13:22.799864 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.799842 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-xtwsb"] Oct 30 13:13:22.834288 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.834262 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn"] Oct 30 13:13:22.840025 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:22.840004 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-bb71e2-openshift-default-7c5c5c884c-jh7gn"] Oct 30 13:13:23.088237 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:23.088203 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2" path="/var/lib/kubelet/pods/17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2/volumes" Oct 30 13:13:23.088678 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:23.088663 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c461dc10-21d9-40ea-ad0b-1125e530b240" path="/var/lib/kubelet/pods/c461dc10-21d9-40ea-ad0b-1125e530b240/volumes" Oct 30 13:13:23.089092 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:23.089060 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb32630c-51c4-495d-8a43-eed34246ada8" path="/var/lib/kubelet/pods/eb32630c-51c4-495d-8a43-eed34246ada8/volumes" Oct 30 13:13:30.741646 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:30.741583 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" podUID="b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" containerName="main" containerID="cri-o://e257543bc51f2701f7c36eb75e5a1c2e53cd0dfeaa769fabed96ade1638368bf" gracePeriod=45 Oct 30 13:13:30.768038 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:30.768006 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" podUID="19409b77-526f-48f7-b539-6d8ebb2d3dfc" containerName="main" containerID="cri-o://3dd6aae5c749df07ad715462c10bdccc1235f5a7fc30caf34e834c7df061bf06" gracePeriod=45 Oct 30 13:13:31.022153 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.022129 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:13:31.035447 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.035425 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:13:31.120233 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.120186 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-dshm\" (UniqueName: \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-dshm\") pod \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " Oct 30 13:13:31.120403 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.120248 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-tmp-dir\") pod \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " Oct 30 13:13:31.120403 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.120278 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-tls-certs\" (UniqueName: \"kubernetes.io/secret/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-tls-certs\") pod \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " Oct 30 13:13:31.120403 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.120298 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-tmp-dir\") pod \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " Oct 30 13:13:31.120403 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.120329 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-kube-api-access-mgxtx\" (UniqueName: \"kubernetes.io/projected/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-kube-api-access-mgxtx\") pod \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " Oct 30 13:13:31.120403 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.120357 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-dshm\") pod \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " Oct 30 13:13:31.120403 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.120402 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19409b77-526f-48f7-b539-6d8ebb2d3dfc-tls-certs\" (UniqueName: \"kubernetes.io/secret/19409b77-526f-48f7-b539-6d8ebb2d3dfc-tls-certs\") pod \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " Oct 30 13:13:31.120664 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.120425 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-home\" (UniqueName: \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-home\") pod \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " Oct 30 13:13:31.120664 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.120469 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19409b77-526f-48f7-b539-6d8ebb2d3dfc-kube-api-access-n4mvr\" (UniqueName: \"kubernetes.io/projected/19409b77-526f-48f7-b539-6d8ebb2d3dfc-kube-api-access-n4mvr\") pod \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " Oct 30 13:13:31.120664 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.120485 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-home\" (UniqueName: \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-home\") pod \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " Oct 30 13:13:31.120664 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.120520 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-model-cache\") pod \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\" (UID: \"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5\") " Oct 30 13:13:31.120664 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.120551 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-model-cache\") pod \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\" (UID: \"19409b77-526f-48f7-b539-6d8ebb2d3dfc\") " Oct 30 13:13:31.122105 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.121147 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-model-cache" pod "19409b77-526f-48f7-b539-6d8ebb2d3dfc" (UID: "19409b77-526f-48f7-b539-6d8ebb2d3dfc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:31.122105 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.121525 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-home" pod "19409b77-526f-48f7-b539-6d8ebb2d3dfc" (UID: "19409b77-526f-48f7-b539-6d8ebb2d3dfc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:31.122105 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.121662 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-tmp-dir" pod "19409b77-526f-48f7-b539-6d8ebb2d3dfc" (UID: "19409b77-526f-48f7-b539-6d8ebb2d3dfc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:31.122105 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.121675 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-home" pod "b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" (UID: "b3f1a5b4-af0c-4ae3-b386-688d0cf924a5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:31.122105 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.121806 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-model-cache" pod "b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" (UID: "b3f1a5b4-af0c-4ae3-b386-688d0cf924a5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:31.122105 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.122037 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-tmp-dir" pod "b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" (UID: "b3f1a5b4-af0c-4ae3-b386-688d0cf924a5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:31.123004 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.122979 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-dshm" pod "19409b77-526f-48f7-b539-6d8ebb2d3dfc" (UID: "19409b77-526f-48f7-b539-6d8ebb2d3dfc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:31.123321 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.123281 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-tls-certs" pod "b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" (UID: "b3f1a5b4-af0c-4ae3-b386-688d0cf924a5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:13:31.123418 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.123365 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19409b77-526f-48f7-b539-6d8ebb2d3dfc-tls-certs" pod "19409b77-526f-48f7-b539-6d8ebb2d3dfc" (UID: "19409b77-526f-48f7-b539-6d8ebb2d3dfc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:13:31.123562 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.123543 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-dshm" pod "b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" (UID: "b3f1a5b4-af0c-4ae3-b386-688d0cf924a5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:31.124775 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.124754 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-kube-api-access-mgxtx" pod "b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" (UID: "b3f1a5b4-af0c-4ae3-b386-688d0cf924a5"). InnerVolumeSpecName "kube-api-access-mgxtx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:31.125369 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.125344 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19409b77-526f-48f7-b539-6d8ebb2d3dfc-kube-api-access-n4mvr" pod "19409b77-526f-48f7-b539-6d8ebb2d3dfc" (UID: "19409b77-526f-48f7-b539-6d8ebb2d3dfc"). InnerVolumeSpecName "kube-api-access-n4mvr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:31.221896 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.221867 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4mvr\" (UniqueName: \"kubernetes.io/projected/19409b77-526f-48f7-b539-6d8ebb2d3dfc-kube-api-access-n4mvr\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:31.221896 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.221893 5029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-home\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:31.221896 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.221905 5029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-model-cache\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:31.222165 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.221920 5029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-model-cache\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:31.222165 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.221929 5029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-dshm\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:31.222165 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.221937 5029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-tmp-dir\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:31.222165 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.221944 5029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-tls-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:31.222165 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.221951 5029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-tmp-dir\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:31.222165 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.221959 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgxtx\" (UniqueName: \"kubernetes.io/projected/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-kube-api-access-mgxtx\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:31.222165 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.221966 5029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5-dshm\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:31.222165 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.221974 5029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19409b77-526f-48f7-b539-6d8ebb2d3dfc-tls-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:31.222165 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.221982 5029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19409b77-526f-48f7-b539-6d8ebb2d3dfc-home\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:31.742939 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.742906 5029 generic.go:356] "Generic (PLEG): container finished" podID="b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" containerID="e257543bc51f2701f7c36eb75e5a1c2e53cd0dfeaa769fabed96ade1638368bf" exitCode=0 Oct 30 13:13:31.743422 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.742951 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" Oct 30 13:13:31.743422 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.742986 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" event={"ID":"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5","Type":"ContainerDied","Data":"e257543bc51f2701f7c36eb75e5a1c2e53cd0dfeaa769fabed96ade1638368bf"} Oct 30 13:13:31.743422 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.743034 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" event={"ID":"b3f1a5b4-af0c-4ae3-b386-688d0cf924a5","Type":"ContainerDied","Data":"db1a649312d9563adbe1ce05e06c2e13c116c70dcef6ec81384b2cd4966894a0"} Oct 30 13:13:31.743422 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.743054 5029 scope.go:122] "RemoveContainer" containerID="e257543bc51f2701f7c36eb75e5a1c2e53cd0dfeaa769fabed96ade1638368bf" Oct 30 13:13:31.744387 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.744367 5029 generic.go:356] "Generic (PLEG): container finished" podID="19409b77-526f-48f7-b539-6d8ebb2d3dfc" containerID="3dd6aae5c749df07ad715462c10bdccc1235f5a7fc30caf34e834c7df061bf06" exitCode=0 Oct 30 13:13:31.744452 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.744418 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" Oct 30 13:13:31.744510 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.744448 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" event={"ID":"19409b77-526f-48f7-b539-6d8ebb2d3dfc","Type":"ContainerDied","Data":"3dd6aae5c749df07ad715462c10bdccc1235f5a7fc30caf34e834c7df061bf06"} Oct 30 13:13:31.744510 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.744489 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p" event={"ID":"19409b77-526f-48f7-b539-6d8ebb2d3dfc","Type":"ContainerDied","Data":"73acb5a510cf98ef6502b8130955b575216fc4598965fd60ee62d18d41700b55"} Oct 30 13:13:31.763680 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.763663 5029 scope.go:122] "RemoveContainer" containerID="e257543bc51f2701f7c36eb75e5a1c2e53cd0dfeaa769fabed96ade1638368bf" Oct 30 13:13:31.763939 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:13:31.763921 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e257543bc51f2701f7c36eb75e5a1c2e53cd0dfeaa769fabed96ade1638368bf\": container with ID starting with e257543bc51f2701f7c36eb75e5a1c2e53cd0dfeaa769fabed96ade1638368bf not found: ID does not exist" containerID="e257543bc51f2701f7c36eb75e5a1c2e53cd0dfeaa769fabed96ade1638368bf" Oct 30 13:13:31.763979 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.763948 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e257543bc51f2701f7c36eb75e5a1c2e53cd0dfeaa769fabed96ade1638368bf"} err="failed to get container status \"e257543bc51f2701f7c36eb75e5a1c2e53cd0dfeaa769fabed96ade1638368bf\": rpc error: code = NotFound desc = could not find container \"e257543bc51f2701f7c36eb75e5a1c2e53cd0dfeaa769fabed96ade1638368bf\": container with ID starting with e257543bc51f2701f7c36eb75e5a1c2e53cd0dfeaa769fabed96ade1638368bf not found: ID does not exist" Oct 30 13:13:31.763979 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.763965 5029 scope.go:122] "RemoveContainer" containerID="3dd6aae5c749df07ad715462c10bdccc1235f5a7fc30caf34e834c7df061bf06" Oct 30 13:13:31.771660 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.771642 5029 scope.go:122] "RemoveContainer" containerID="3dd6aae5c749df07ad715462c10bdccc1235f5a7fc30caf34e834c7df061bf06" Oct 30 13:13:31.771905 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:13:31.771886 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dd6aae5c749df07ad715462c10bdccc1235f5a7fc30caf34e834c7df061bf06\": container with ID starting with 3dd6aae5c749df07ad715462c10bdccc1235f5a7fc30caf34e834c7df061bf06 not found: ID does not exist" containerID="3dd6aae5c749df07ad715462c10bdccc1235f5a7fc30caf34e834c7df061bf06" Oct 30 13:13:31.772002 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.771908 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dd6aae5c749df07ad715462c10bdccc1235f5a7fc30caf34e834c7df061bf06"} err="failed to get container status \"3dd6aae5c749df07ad715462c10bdccc1235f5a7fc30caf34e834c7df061bf06\": rpc error: code = NotFound desc = could not find container \"3dd6aae5c749df07ad715462c10bdccc1235f5a7fc30caf34e834c7df061bf06\": container with ID starting with 3dd6aae5c749df07ad715462c10bdccc1235f5a7fc30caf34e834c7df061bf06 not found: ID does not exist" Oct 30 13:13:31.808567 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.808542 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p"] Oct 30 13:13:31.819881 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.819851 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-175831/simulator-w5-175831-kserve-5b7646f447-7p64p"] Oct 30 13:13:31.847876 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.847852 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg"] Oct 30 13:13:31.850936 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.850916 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg"] Oct 30 13:13:31.952334 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:31.952286 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-175831/premium-w5-175831-kserve-6cc7d45586-fwrbg" podUID="b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 13:13:33.088788 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:33.088753 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="19409b77-526f-48f7-b539-6d8ebb2d3dfc" path="/var/lib/kubelet/pods/19409b77-526f-48f7-b539-6d8ebb2d3dfc/volumes" Oct 30 13:13:33.089237 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:33.089222 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" path="/var/lib/kubelet/pods/b3f1a5b4-af0c-4ae3-b386-688d0cf924a5/volumes" Oct 30 13:13:41.461365 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:41.461334 5029 scope.go:122] "RemoveContainer" containerID="c142c44181093d993c3ab7115ed95392d6a049e2e7a2c9f2bbb18a7bfd434e31" Oct 30 13:13:44.880582 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:13:44.880547 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:13:45.100837 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:45.100799 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw"] Oct 30 13:13:45.101079 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:45.101036 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" podUID="499f565c-be4d-4573-ab85-d78f68bf43b7" containerName="istio-proxy" containerID="cri-o://ac5ddf68af04694bfdc3f310b914ed2f451daa3748bd3cb5853a8a630cdbf723" gracePeriod=30 Oct 30 13:13:45.112718 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:45.112682 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq"] Oct 30 13:13:45.112925 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:45.112894 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" podUID="819605a7-0156-4e46-922c-26d314885598" containerName="istio-proxy" containerID="cri-o://8692fc5e14564eeb40853787bd8c729343ddbb4afbf0856cddad3e3a0b1b6667" gracePeriod=30 Oct 30 13:13:45.132940 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:45.132861 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69"] Oct 30 13:13:45.133135 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:45.133107 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" podUID="6e53092c-b84c-406f-badd-976aba85f453" containerName="istio-proxy" containerID="cri-o://b05607f2471236a1a04b70cdfd4fd1447766daefbf1e4b0fe7fdc2becb28ee30" gracePeriod=30 Oct 30 13:13:50.415370 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.415340 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:13:50.444811 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.444786 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:13:50.449823 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.449797 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:13:50.531911 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.531878 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-data\") pod \"499f565c-be4d-4573-ab85-d78f68bf43b7\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " Oct 30 13:13:50.532120 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.531928 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/819605a7-0156-4e46-922c-26d314885598-kube-api-access-glrn4\" (UniqueName: \"kubernetes.io/projected/819605a7-0156-4e46-922c-26d314885598-kube-api-access-glrn4\") pod \"819605a7-0156-4e46-922c-26d314885598\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " Oct 30 13:13:50.532120 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.531956 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-istio-data\") pod \"6e53092c-b84c-406f-badd-976aba85f453\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " Oct 30 13:13:50.532120 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.531982 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/499f565c-be4d-4573-ab85-d78f68bf43b7-kube-api-access-mj9hw\" (UniqueName: \"kubernetes.io/projected/499f565c-be4d-4573-ab85-d78f68bf43b7-kube-api-access-mj9hw\") pod \"499f565c-be4d-4573-ab85-d78f68bf43b7\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " Oct 30 13:13:50.532120 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532003 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-envoy\") pod \"499f565c-be4d-4573-ab85-d78f68bf43b7\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " Oct 30 13:13:50.532120 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532018 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-workload-certs\") pod \"499f565c-be4d-4573-ab85-d78f68bf43b7\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " Oct 30 13:13:50.532120 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532039 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-credential-socket\") pod \"499f565c-be4d-4573-ab85-d78f68bf43b7\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " Oct 30 13:13:50.532120 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532092 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-istio-data\") pod \"819605a7-0156-4e46-922c-26d314885598\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " Oct 30 13:13:50.532120 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532123 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-credential-socket\") pod \"819605a7-0156-4e46-922c-26d314885598\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " Oct 30 13:13:50.532575 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532141 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/819605a7-0156-4e46-922c-26d314885598-istio-token\" (UniqueName: \"kubernetes.io/projected/819605a7-0156-4e46-922c-26d314885598-istio-token\") pod \"819605a7-0156-4e46-922c-26d314885598\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " Oct 30 13:13:50.532575 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532175 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6e53092c-b84c-406f-badd-976aba85f453-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e53092c-b84c-406f-badd-976aba85f453-istiod-ca-cert\") pod \"6e53092c-b84c-406f-badd-976aba85f453\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " Oct 30 13:13:50.532575 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532204 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-workload-certs\") pod \"819605a7-0156-4e46-922c-26d314885598\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " Oct 30 13:13:50.532575 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532224 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-data" pod "499f565c-be4d-4573-ab85-d78f68bf43b7" (UID: "499f565c-be4d-4573-ab85-d78f68bf43b7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:50.532575 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532244 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e53092c-b84c-406f-badd-976aba85f453-kube-api-access-d5sn5\" (UniqueName: \"kubernetes.io/projected/6e53092c-b84c-406f-badd-976aba85f453-kube-api-access-d5sn5\") pod \"6e53092c-b84c-406f-badd-976aba85f453\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " Oct 30 13:13:50.532575 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532282 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-workload-socket\") pod \"819605a7-0156-4e46-922c-26d314885598\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " Oct 30 13:13:50.532575 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532308 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6e53092c-b84c-406f-badd-976aba85f453-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e53092c-b84c-406f-badd-976aba85f453-istio-podinfo\") pod \"6e53092c-b84c-406f-badd-976aba85f453\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " Oct 30 13:13:50.532575 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532335 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-istio-envoy\") pod \"6e53092c-b84c-406f-badd-976aba85f453\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " Oct 30 13:13:50.532575 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532367 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/499f565c-be4d-4573-ab85-d78f68bf43b7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/499f565c-be4d-4573-ab85-d78f68bf43b7-istiod-ca-cert\") pod \"499f565c-be4d-4573-ab85-d78f68bf43b7\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " Oct 30 13:13:50.532575 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532398 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-istio-envoy\") pod \"819605a7-0156-4e46-922c-26d314885598\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " Oct 30 13:13:50.532575 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532416 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-workload-certs\") pod \"6e53092c-b84c-406f-badd-976aba85f453\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " Oct 30 13:13:50.532575 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532434 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-credential-socket" pod "499f565c-be4d-4573-ab85-d78f68bf43b7" (UID: "499f565c-be4d-4573-ab85-d78f68bf43b7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:50.532575 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532434 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-token\" (UniqueName: \"kubernetes.io/projected/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-token\") pod \"499f565c-be4d-4573-ab85-d78f68bf43b7\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " Oct 30 13:13:50.532575 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532474 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-workload-socket\") pod \"499f565c-be4d-4573-ab85-d78f68bf43b7\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " Oct 30 13:13:50.532575 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532497 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/819605a7-0156-4e46-922c-26d314885598-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/819605a7-0156-4e46-922c-26d314885598-istiod-ca-cert\") pod \"819605a7-0156-4e46-922c-26d314885598\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " Oct 30 13:13:50.532575 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532526 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-workload-socket\") pod \"6e53092c-b84c-406f-badd-976aba85f453\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " Oct 30 13:13:50.532575 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532543 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e53092c-b84c-406f-badd-976aba85f453-istio-token\" (UniqueName: \"kubernetes.io/projected/6e53092c-b84c-406f-badd-976aba85f453-istio-token\") pod \"6e53092c-b84c-406f-badd-976aba85f453\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " Oct 30 13:13:50.532575 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532582 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-credential-socket\") pod \"6e53092c-b84c-406f-badd-976aba85f453\" (UID: \"6e53092c-b84c-406f-badd-976aba85f453\") " Oct 30 13:13:50.533485 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532605 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/819605a7-0156-4e46-922c-26d314885598-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/819605a7-0156-4e46-922c-26d314885598-istio-podinfo\") pod \"819605a7-0156-4e46-922c-26d314885598\" (UID: \"819605a7-0156-4e46-922c-26d314885598\") " Oct 30 13:13:50.533485 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.532631 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-podinfo\") pod \"499f565c-be4d-4573-ab85-d78f68bf43b7\" (UID: \"499f565c-be4d-4573-ab85-d78f68bf43b7\") " Oct 30 13:13:50.533485 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.533043 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-data\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.533485 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.533087 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-credential-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.533485 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.533136 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/499f565c-be4d-4573-ab85-d78f68bf43b7-istiod-ca-cert" pod "499f565c-be4d-4573-ab85-d78f68bf43b7" (UID: "499f565c-be4d-4573-ab85-d78f68bf43b7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:13:50.533485 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.533447 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-istio-data" pod "819605a7-0156-4e46-922c-26d314885598" (UID: "819605a7-0156-4e46-922c-26d314885598"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:50.533781 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.533636 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-credential-socket" pod "819605a7-0156-4e46-922c-26d314885598" (UID: "819605a7-0156-4e46-922c-26d314885598"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:50.535025 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.534316 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/819605a7-0156-4e46-922c-26d314885598-istiod-ca-cert" pod "819605a7-0156-4e46-922c-26d314885598" (UID: "819605a7-0156-4e46-922c-26d314885598"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:13:50.535025 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.534430 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-workload-certs" pod "819605a7-0156-4e46-922c-26d314885598" (UID: "819605a7-0156-4e46-922c-26d314885598"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:50.535025 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.534556 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-workload-certs" pod "6e53092c-b84c-406f-badd-976aba85f453" (UID: "6e53092c-b84c-406f-badd-976aba85f453"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:50.535025 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.534640 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e53092c-b84c-406f-badd-976aba85f453-istiod-ca-cert" pod "6e53092c-b84c-406f-badd-976aba85f453" (UID: "6e53092c-b84c-406f-badd-976aba85f453"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:13:50.535025 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.534764 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-workload-socket" pod "499f565c-be4d-4573-ab85-d78f68bf43b7" (UID: "499f565c-be4d-4573-ab85-d78f68bf43b7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:50.535025 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.534808 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-istio-data" pod "6e53092c-b84c-406f-badd-976aba85f453" (UID: "6e53092c-b84c-406f-badd-976aba85f453"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:50.535025 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.534977 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-workload-socket" pod "819605a7-0156-4e46-922c-26d314885598" (UID: "819605a7-0156-4e46-922c-26d314885598"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:50.535485 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.535179 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-workload-certs" pod "499f565c-be4d-4573-ab85-d78f68bf43b7" (UID: "499f565c-be4d-4573-ab85-d78f68bf43b7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:50.535485 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.535358 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-workload-socket" pod "6e53092c-b84c-406f-badd-976aba85f453" (UID: "6e53092c-b84c-406f-badd-976aba85f453"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:50.535485 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.535441 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/499f565c-be4d-4573-ab85-d78f68bf43b7-kube-api-access-mj9hw" pod "499f565c-be4d-4573-ab85-d78f68bf43b7" (UID: "499f565c-be4d-4573-ab85-d78f68bf43b7"). InnerVolumeSpecName "kube-api-access-mj9hw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:50.535663 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.535638 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-credential-socket" pod "6e53092c-b84c-406f-badd-976aba85f453" (UID: "6e53092c-b84c-406f-badd-976aba85f453"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:50.537956 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.537907 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/819605a7-0156-4e46-922c-26d314885598-istio-podinfo" pod "819605a7-0156-4e46-922c-26d314885598" (UID: "819605a7-0156-4e46-922c-26d314885598"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 13:13:50.538179 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.538145 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e53092c-b84c-406f-badd-976aba85f453-istio-token" pod "6e53092c-b84c-406f-badd-976aba85f453" (UID: "6e53092c-b84c-406f-badd-976aba85f453"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:50.538587 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.538538 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/819605a7-0156-4e46-922c-26d314885598-kube-api-access-glrn4" pod "819605a7-0156-4e46-922c-26d314885598" (UID: "819605a7-0156-4e46-922c-26d314885598"). InnerVolumeSpecName "kube-api-access-glrn4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:50.538935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.538899 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-istio-envoy" pod "6e53092c-b84c-406f-badd-976aba85f453" (UID: "6e53092c-b84c-406f-badd-976aba85f453"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:50.543164 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.543119 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6e53092c-b84c-406f-badd-976aba85f453-istio-podinfo" pod "6e53092c-b84c-406f-badd-976aba85f453" (UID: "6e53092c-b84c-406f-badd-976aba85f453"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 13:13:50.543390 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.543356 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e53092c-b84c-406f-badd-976aba85f453-kube-api-access-d5sn5" pod "6e53092c-b84c-406f-badd-976aba85f453" (UID: "6e53092c-b84c-406f-badd-976aba85f453"). InnerVolumeSpecName "kube-api-access-d5sn5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:50.543621 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.543585 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/819605a7-0156-4e46-922c-26d314885598-istio-token" pod "819605a7-0156-4e46-922c-26d314885598" (UID: "819605a7-0156-4e46-922c-26d314885598"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:50.543845 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.543809 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-envoy" pod "499f565c-be4d-4573-ab85-d78f68bf43b7" (UID: "499f565c-be4d-4573-ab85-d78f68bf43b7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:50.543920 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.543844 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-istio-envoy" pod "819605a7-0156-4e46-922c-26d314885598" (UID: "819605a7-0156-4e46-922c-26d314885598"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:13:50.544060 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.544037 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-podinfo" pod "499f565c-be4d-4573-ab85-d78f68bf43b7" (UID: "499f565c-be4d-4573-ab85-d78f68bf43b7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 13:13:50.545699 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.545676 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-token" pod "499f565c-be4d-4573-ab85-d78f68bf43b7" (UID: "499f565c-be4d-4573-ab85-d78f68bf43b7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:13:50.634623 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634573 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glrn4\" (UniqueName: \"kubernetes.io/projected/819605a7-0156-4e46-922c-26d314885598-kube-api-access-glrn4\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634623 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634619 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-istio-data\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634635 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mj9hw\" (UniqueName: \"kubernetes.io/projected/499f565c-be4d-4573-ab85-d78f68bf43b7-kube-api-access-mj9hw\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634649 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-envoy\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634663 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-workload-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634675 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-istio-data\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634688 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-credential-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634704 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/819605a7-0156-4e46-922c-26d314885598-istio-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634719 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e53092c-b84c-406f-badd-976aba85f453-istiod-ca-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634733 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-workload-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634747 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5sn5\" (UniqueName: \"kubernetes.io/projected/6e53092c-b84c-406f-badd-976aba85f453-kube-api-access-d5sn5\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634755 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-workload-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634764 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e53092c-b84c-406f-badd-976aba85f453-istio-podinfo\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634772 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-istio-envoy\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634782 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/499f565c-be4d-4573-ab85-d78f68bf43b7-istiod-ca-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634796 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/819605a7-0156-4e46-922c-26d314885598-istio-envoy\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634806 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-workload-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634814 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634822 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/499f565c-be4d-4573-ab85-d78f68bf43b7-workload-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634829 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/819605a7-0156-4e46-922c-26d314885598-istiod-ca-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634837 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-workload-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634848 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e53092c-b84c-406f-badd-976aba85f453-istio-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634856 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e53092c-b84c-406f-badd-976aba85f453-credential-socket\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634864 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/819605a7-0156-4e46-922c-26d314885598-istio-podinfo\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.634877 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.634873 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/499f565c-be4d-4573-ab85-d78f68bf43b7-istio-podinfo\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:13:50.839756 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.839709 5029 generic.go:356] "Generic (PLEG): container finished" podID="819605a7-0156-4e46-922c-26d314885598" containerID="8692fc5e14564eeb40853787bd8c729343ddbb4afbf0856cddad3e3a0b1b6667" exitCode=0 Oct 30 13:13:50.839756 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.839756 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" Oct 30 13:13:50.840015 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.839795 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" event={"ID":"819605a7-0156-4e46-922c-26d314885598","Type":"ContainerDied","Data":"8692fc5e14564eeb40853787bd8c729343ddbb4afbf0856cddad3e3a0b1b6667"} Oct 30 13:13:50.840015 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.839865 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq" event={"ID":"819605a7-0156-4e46-922c-26d314885598","Type":"ContainerDied","Data":"c9107d3c665455b2619fa74feb0ac60ca17c775810c5aaec73c2473d98be6440"} Oct 30 13:13:50.840015 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.839915 5029 scope.go:122] "RemoveContainer" containerID="8692fc5e14564eeb40853787bd8c729343ddbb4afbf0856cddad3e3a0b1b6667" Oct 30 13:13:50.842239 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.842175 5029 generic.go:356] "Generic (PLEG): container finished" podID="499f565c-be4d-4573-ab85-d78f68bf43b7" containerID="ac5ddf68af04694bfdc3f310b914ed2f451daa3748bd3cb5853a8a630cdbf723" exitCode=0 Oct 30 13:13:50.842373 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.842309 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" event={"ID":"499f565c-be4d-4573-ab85-d78f68bf43b7","Type":"ContainerDied","Data":"ac5ddf68af04694bfdc3f310b914ed2f451daa3748bd3cb5853a8a630cdbf723"} Oct 30 13:13:50.842373 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.842354 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" event={"ID":"499f565c-be4d-4573-ab85-d78f68bf43b7","Type":"ContainerDied","Data":"54aa88c734bb42914d36abb2679c0412b99c5dea8474492c921fbe7c98574d07"} Oct 30 13:13:50.842499 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.842429 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw" Oct 30 13:13:50.847908 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.847777 5029 generic.go:356] "Generic (PLEG): container finished" podID="6e53092c-b84c-406f-badd-976aba85f453" containerID="b05607f2471236a1a04b70cdfd4fd1447766daefbf1e4b0fe7fdc2becb28ee30" exitCode=0 Oct 30 13:13:50.847908 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.847827 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" Oct 30 13:13:50.847908 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.847828 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" event={"ID":"6e53092c-b84c-406f-badd-976aba85f453","Type":"ContainerDied","Data":"b05607f2471236a1a04b70cdfd4fd1447766daefbf1e4b0fe7fdc2becb28ee30"} Oct 30 13:13:50.847908 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.847876 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69" event={"ID":"6e53092c-b84c-406f-badd-976aba85f453","Type":"ContainerDied","Data":"8d27bb19b0ffb05c530f3b74e6a05b853b19760d0fdbb31cc4571e48b199ba2c"} Oct 30 13:13:50.854298 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.854225 5029 scope.go:122] "RemoveContainer" containerID="8692fc5e14564eeb40853787bd8c729343ddbb4afbf0856cddad3e3a0b1b6667" Oct 30 13:13:50.854751 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:13:50.854724 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8692fc5e14564eeb40853787bd8c729343ddbb4afbf0856cddad3e3a0b1b6667\": container with ID starting with 8692fc5e14564eeb40853787bd8c729343ddbb4afbf0856cddad3e3a0b1b6667 not found: ID does not exist" containerID="8692fc5e14564eeb40853787bd8c729343ddbb4afbf0856cddad3e3a0b1b6667" Oct 30 13:13:50.854841 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.854762 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8692fc5e14564eeb40853787bd8c729343ddbb4afbf0856cddad3e3a0b1b6667"} err="failed to get container status \"8692fc5e14564eeb40853787bd8c729343ddbb4afbf0856cddad3e3a0b1b6667\": rpc error: code = NotFound desc = could not find container \"8692fc5e14564eeb40853787bd8c729343ddbb4afbf0856cddad3e3a0b1b6667\": container with ID starting with 8692fc5e14564eeb40853787bd8c729343ddbb4afbf0856cddad3e3a0b1b6667 not found: ID does not exist" Oct 30 13:13:50.854841 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.854790 5029 scope.go:122] "RemoveContainer" containerID="ac5ddf68af04694bfdc3f310b914ed2f451daa3748bd3cb5853a8a630cdbf723" Oct 30 13:13:50.871943 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.871927 5029 scope.go:122] "RemoveContainer" containerID="ac5ddf68af04694bfdc3f310b914ed2f451daa3748bd3cb5853a8a630cdbf723" Oct 30 13:13:50.872261 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:13:50.872242 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac5ddf68af04694bfdc3f310b914ed2f451daa3748bd3cb5853a8a630cdbf723\": container with ID starting with ac5ddf68af04694bfdc3f310b914ed2f451daa3748bd3cb5853a8a630cdbf723 not found: ID does not exist" containerID="ac5ddf68af04694bfdc3f310b914ed2f451daa3748bd3cb5853a8a630cdbf723" Oct 30 13:13:50.872343 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.872274 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac5ddf68af04694bfdc3f310b914ed2f451daa3748bd3cb5853a8a630cdbf723"} err="failed to get container status \"ac5ddf68af04694bfdc3f310b914ed2f451daa3748bd3cb5853a8a630cdbf723\": rpc error: code = NotFound desc = could not find container \"ac5ddf68af04694bfdc3f310b914ed2f451daa3748bd3cb5853a8a630cdbf723\": container with ID starting with ac5ddf68af04694bfdc3f310b914ed2f451daa3748bd3cb5853a8a630cdbf723 not found: ID does not exist" Oct 30 13:13:50.872343 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.872296 5029 scope.go:122] "RemoveContainer" containerID="b05607f2471236a1a04b70cdfd4fd1447766daefbf1e4b0fe7fdc2becb28ee30" Oct 30 13:13:50.882696 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.882677 5029 scope.go:122] "RemoveContainer" containerID="b05607f2471236a1a04b70cdfd4fd1447766daefbf1e4b0fe7fdc2becb28ee30" Oct 30 13:13:50.882975 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:13:50.882956 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b05607f2471236a1a04b70cdfd4fd1447766daefbf1e4b0fe7fdc2becb28ee30\": container with ID starting with b05607f2471236a1a04b70cdfd4fd1447766daefbf1e4b0fe7fdc2becb28ee30 not found: ID does not exist" containerID="b05607f2471236a1a04b70cdfd4fd1447766daefbf1e4b0fe7fdc2becb28ee30" Oct 30 13:13:50.883042 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.882985 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b05607f2471236a1a04b70cdfd4fd1447766daefbf1e4b0fe7fdc2becb28ee30"} err="failed to get container status \"b05607f2471236a1a04b70cdfd4fd1447766daefbf1e4b0fe7fdc2becb28ee30\": rpc error: code = NotFound desc = could not find container \"b05607f2471236a1a04b70cdfd4fd1447766daefbf1e4b0fe7fdc2becb28ee30\": container with ID starting with b05607f2471236a1a04b70cdfd4fd1447766daefbf1e4b0fe7fdc2becb28ee30 not found: ID does not exist" Oct 30 13:13:50.892393 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.892374 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq"] Oct 30 13:13:50.903648 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.903628 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-dgzgq"] Oct 30 13:13:50.960516 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.960480 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69"] Oct 30 13:13:50.965139 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.965113 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-zkq69"] Oct 30 13:13:50.998813 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:50.998778 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw"] Oct 30 13:13:51.006884 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:51.006854 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-175831-openshift-default-686b69df8f-rzdzw"] Oct 30 13:13:51.089108 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:51.089057 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="499f565c-be4d-4573-ab85-d78f68bf43b7" path="/var/lib/kubelet/pods/499f565c-be4d-4573-ab85-d78f68bf43b7/volumes" Oct 30 13:13:51.089601 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:51.089580 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e53092c-b84c-406f-badd-976aba85f453" path="/var/lib/kubelet/pods/6e53092c-b84c-406f-badd-976aba85f453/volumes" Oct 30 13:13:51.090095 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:13:51.090030 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="819605a7-0156-4e46-922c-26d314885598" path="/var/lib/kubelet/pods/819605a7-0156-4e46-922c-26d314885598/volumes" Oct 30 13:14:14.879180 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:14:14.879145 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:14:26.173424 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.173391 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8"] Oct 30 13:14:26.173842 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.173791 5029 cpu_manager.go:414] "containerMap: removing container" podUID="d6bdcd3f-0200-48e6-af85-2d1d8f022054" containerName="main" Oct 30 13:14:26.173842 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.173806 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6bdcd3f-0200-48e6-af85-2d1d8f022054" containerName="main" Oct 30 13:14:26.173842 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.173818 5029 cpu_manager.go:414] "containerMap: removing container" podUID="c461dc10-21d9-40ea-ad0b-1125e530b240" containerName="istio-proxy" Oct 30 13:14:26.173842 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.173825 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c461dc10-21d9-40ea-ad0b-1125e530b240" containerName="istio-proxy" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.173844 5029 cpu_manager.go:414] "containerMap: removing container" podUID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" containerName="main" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.173852 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" containerName="main" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.173865 5029 cpu_manager.go:414] "containerMap: removing container" podUID="2b878b27-465c-433d-aba4-e31849cfc14c" containerName="main" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.173873 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b878b27-465c-433d-aba4-e31849cfc14c" containerName="main" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.173886 5029 cpu_manager.go:414] "containerMap: removing container" podUID="6e53092c-b84c-406f-badd-976aba85f453" containerName="istio-proxy" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.173893 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e53092c-b84c-406f-badd-976aba85f453" containerName="istio-proxy" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.173905 5029 cpu_manager.go:414] "containerMap: removing container" podUID="a2dc4823-8550-4c35-a226-6440fbbffbc5" containerName="istio-proxy" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.173912 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2dc4823-8550-4c35-a226-6440fbbffbc5" containerName="istio-proxy" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.173933 5029 cpu_manager.go:414] "containerMap: removing container" podUID="48dfe4a1-9d55-4f04-812a-1d0084f01c7e" containerName="istio-proxy" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.173940 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48dfe4a1-9d55-4f04-812a-1d0084f01c7e" containerName="istio-proxy" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.173972 5029 cpu_manager.go:414] "containerMap: removing container" podUID="0e23ddba-6739-4ccb-803a-d19cb9501b95" containerName="maas-api" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.173980 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e23ddba-6739-4ccb-803a-d19cb9501b95" containerName="maas-api" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.173990 5029 cpu_manager.go:414] "containerMap: removing container" podUID="06d7d761-d47f-4aa5-b931-46f910b65597" containerName="main" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.173998 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06d7d761-d47f-4aa5-b931-46f910b65597" containerName="main" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174006 5029 cpu_manager.go:414] "containerMap: removing container" podUID="b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" containerName="main" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174012 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" containerName="main" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174034 5029 cpu_manager.go:414] "containerMap: removing container" podUID="60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4" containerName="maas-api" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174041 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4" containerName="maas-api" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174055 5029 cpu_manager.go:414] "containerMap: removing container" podUID="0840bdc7-256c-4b2b-a76a-8d5063993d00" containerName="istio-proxy" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174087 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0840bdc7-256c-4b2b-a76a-8d5063993d00" containerName="istio-proxy" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174101 5029 cpu_manager.go:414] "containerMap: removing container" podUID="499f565c-be4d-4573-ab85-d78f68bf43b7" containerName="istio-proxy" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174108 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="499f565c-be4d-4573-ab85-d78f68bf43b7" containerName="istio-proxy" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174117 5029 cpu_manager.go:414] "containerMap: removing container" podUID="efa3fa51-d5f3-49c2-9658-378b1a6a2719" containerName="istio-proxy" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174124 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="efa3fa51-d5f3-49c2-9658-378b1a6a2719" containerName="istio-proxy" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174137 5029 cpu_manager.go:414] "containerMap: removing container" podUID="eb32630c-51c4-495d-8a43-eed34246ada8" containerName="istio-proxy" Oct 30 13:14:26.174134 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174144 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb32630c-51c4-495d-8a43-eed34246ada8" containerName="istio-proxy" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174158 5029 cpu_manager.go:414] "containerMap: removing container" podUID="17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2" containerName="istio-proxy" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174165 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2" containerName="istio-proxy" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174176 5029 cpu_manager.go:414] "containerMap: removing container" podUID="819605a7-0156-4e46-922c-26d314885598" containerName="istio-proxy" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174183 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="819605a7-0156-4e46-922c-26d314885598" containerName="istio-proxy" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174200 5029 cpu_manager.go:414] "containerMap: removing container" podUID="19409b77-526f-48f7-b539-6d8ebb2d3dfc" containerName="main" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174207 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19409b77-526f-48f7-b539-6d8ebb2d3dfc" containerName="main" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174323 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb32630c-51c4-495d-8a43-eed34246ada8" containerName="istio-proxy" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174338 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="17f3c759-be0a-4f0d-b78a-0e4cbe8adfa2" containerName="istio-proxy" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174345 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="819605a7-0156-4e46-922c-26d314885598" containerName="istio-proxy" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174351 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="19409b77-526f-48f7-b539-6d8ebb2d3dfc" containerName="main" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174357 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e23ddba-6739-4ccb-803a-d19cb9501b95" containerName="maas-api" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174362 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6bdcd3f-0200-48e6-af85-2d1d8f022054" containerName="main" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174367 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3f1a5b4-af0c-4ae3-b386-688d0cf924a5" containerName="main" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174371 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="c461dc10-21d9-40ea-ad0b-1125e530b240" containerName="istio-proxy" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174377 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="06d7d761-d47f-4aa5-b931-46f910b65597" containerName="main" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174384 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="60a01580-b95c-44c9-ae4a-8e0c7d3cb2c4" containerName="maas-api" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174389 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd68fdee-30c0-42ac-bf6e-aff5d2abb46d" containerName="main" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174393 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b878b27-465c-433d-aba4-e31849cfc14c" containerName="main" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174397 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="0840bdc7-256c-4b2b-a76a-8d5063993d00" containerName="istio-proxy" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174402 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e53092c-b84c-406f-badd-976aba85f453" containerName="istio-proxy" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174407 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2dc4823-8550-4c35-a226-6440fbbffbc5" containerName="istio-proxy" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174411 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="499f565c-be4d-4573-ab85-d78f68bf43b7" containerName="istio-proxy" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174416 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="efa3fa51-d5f3-49c2-9658-378b1a6a2719" containerName="istio-proxy" Oct 30 13:14:26.174935 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.174421 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="48dfe4a1-9d55-4f04-812a-1d0084f01c7e" containerName="istio-proxy" Oct 30 13:14:26.177614 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.177595 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.208954 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.208921 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8"] Oct 30 13:14:26.265380 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.265350 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11ce4393-c9ba-4c45-a9e8-276e334f6be0-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.265493 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.265386 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11ce4393-c9ba-4c45-a9e8-276e334f6be0-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.265493 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.265468 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11ce4393-c9ba-4c45-a9e8-276e334f6be0-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.265614 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.265502 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11ce4393-c9ba-4c45-a9e8-276e334f6be0-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.265614 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.265528 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11ce4393-c9ba-4c45-a9e8-276e334f6be0-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.265614 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.265545 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11ce4393-c9ba-4c45-a9e8-276e334f6be0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.265777 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.265672 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11ce4393-c9ba-4c45-a9e8-276e334f6be0-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.265777 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.265705 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tts58\" (UniqueName: \"kubernetes.io/projected/11ce4393-c9ba-4c45-a9e8-276e334f6be0-kube-api-access-tts58\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.265891 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.265807 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11ce4393-c9ba-4c45-a9e8-276e334f6be0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.366875 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.366844 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11ce4393-c9ba-4c45-a9e8-276e334f6be0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.367039 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.366891 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11ce4393-c9ba-4c45-a9e8-276e334f6be0-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.367039 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.367012 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11ce4393-c9ba-4c45-a9e8-276e334f6be0-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.367189 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.367095 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11ce4393-c9ba-4c45-a9e8-276e334f6be0-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.367189 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.367126 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11ce4393-c9ba-4c45-a9e8-276e334f6be0-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.367292 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.367232 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11ce4393-c9ba-4c45-a9e8-276e334f6be0-workload-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.367292 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.367271 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11ce4393-c9ba-4c45-a9e8-276e334f6be0-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.367405 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.367299 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11ce4393-c9ba-4c45-a9e8-276e334f6be0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.367405 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.367327 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11ce4393-c9ba-4c45-a9e8-276e334f6be0-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.367405 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.367356 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tts58\" (UniqueName: \"kubernetes.io/projected/11ce4393-c9ba-4c45-a9e8-276e334f6be0-kube-api-access-tts58\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.367405 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.367363 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11ce4393-c9ba-4c45-a9e8-276e334f6be0-credential-socket\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.367589 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.367408 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11ce4393-c9ba-4c45-a9e8-276e334f6be0-workload-certs\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.367646 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.367620 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11ce4393-c9ba-4c45-a9e8-276e334f6be0-istio-data\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.368032 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.368007 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11ce4393-c9ba-4c45-a9e8-276e334f6be0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.369318 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.369297 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11ce4393-c9ba-4c45-a9e8-276e334f6be0-istio-envoy\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.369443 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.369426 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11ce4393-c9ba-4c45-a9e8-276e334f6be0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.390969 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.390938 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11ce4393-c9ba-4c45-a9e8-276e334f6be0-istio-token\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.391102 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.391083 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tts58\" (UniqueName: \"kubernetes.io/projected/11ce4393-c9ba-4c45-a9e8-276e334f6be0-kube-api-access-tts58\") pod \"maas-default-gateway-openshift-default-5bc884dd87-9h8z8\" (UID: \"11ce4393-c9ba-4c45-a9e8-276e334f6be0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.491838 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.491761 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:26.843279 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.843245 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8"] Oct 30 13:14:26.847485 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:14:26.847453 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11ce4393_c9ba_4c45_a9e8_276e334f6be0.slice/crio-456a28cc2c7baeb77779b0924588c476bc94a1d8a87132d71bac76130a254929 WatchSource:0}: Error finding container 456a28cc2c7baeb77779b0924588c476bc94a1d8a87132d71bac76130a254929: Status 404 returned error can't find the container with id 456a28cc2c7baeb77779b0924588c476bc94a1d8a87132d71bac76130a254929 Oct 30 13:14:26.849451 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.849418 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:14:26.849512 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.849484 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:14:26.849548 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:26.849510 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 30 13:14:27.022144 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:27.022099 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" event={"ID":"11ce4393-c9ba-4c45-a9e8-276e334f6be0","Type":"ContainerStarted","Data":"72eed2094d042c97867160fdba519f78aaadadcf7bd32766ce7f788760e9cca3"} Oct 30 13:14:27.022341 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:27.022160 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" event={"ID":"11ce4393-c9ba-4c45-a9e8-276e334f6be0","Type":"ContainerStarted","Data":"456a28cc2c7baeb77779b0924588c476bc94a1d8a87132d71bac76130a254929"} Oct 30 13:14:27.069432 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:27.069370 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" podStartSLOduration=1.069353514 podStartE2EDuration="1.069353514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:14:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:14:27.067196069 +0000 UTC m=+1366.092837353" watchObservedRunningTime="2026-10-30 13:14:27.069353514 +0000 UTC m=+1366.094994762" Oct 30 13:14:27.495274 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:27.495236 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:27.497155 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:27.497123 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-9h8z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body= Oct 30 13:14:27.497309 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:27.497184 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" podUID="11ce4393-c9ba-4c45-a9e8-276e334f6be0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" Oct 30 13:14:28.495452 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:28.495416 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-9h8z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body= Oct 30 13:14:28.495858 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:28.495472 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" podUID="11ce4393-c9ba-4c45-a9e8-276e334f6be0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" Oct 30 13:14:29.495123 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:29.492299 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-9h8z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body= Oct 30 13:14:29.495123 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:29.492361 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" podUID="11ce4393-c9ba-4c45-a9e8-276e334f6be0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" Oct 30 13:14:30.492571 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:30.492531 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-9h8z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body= Oct 30 13:14:30.493209 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:30.493176 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" podUID="11ce4393-c9ba-4c45-a9e8-276e334f6be0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" Oct 30 13:14:31.495090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:31.494420 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-9h8z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body= Oct 30 13:14:31.495090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:31.494481 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" podUID="11ce4393-c9ba-4c45-a9e8-276e334f6be0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" Oct 30 13:14:32.495458 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:32.495418 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-9h8z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body= Oct 30 13:14:32.495860 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:32.495476 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" podUID="11ce4393-c9ba-4c45-a9e8-276e334f6be0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" Oct 30 13:14:33.493149 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:33.493116 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-9h8z8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body= Oct 30 13:14:33.493324 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:33.493169 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" podUID="11ce4393-c9ba-4c45-a9e8-276e334f6be0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" Oct 30 13:14:35.077527 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:35.077493 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:35.077983 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:35.077788 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:35.078864 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:14:35.078847 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" Oct 30 13:14:44.878177 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:14:44.878143 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:15:14.880766 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:15:14.880679 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:15:21.124569 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:15:21.124512 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-9h8z8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 13:15:21.125169 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:15:21.124604 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9h8z8" podUID="11ce4393-c9ba-4c45-a9e8-276e334f6be0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 13:15:44.878022 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:15:44.877984 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:16:14.878387 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:16:14.878351 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:16:16.436991 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.436949 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc"] Oct 30 13:16:16.437455 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.437243 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" podUID="9b89f6e6-34f4-4888-bea4-26ed65d618e2" containerName="discovery" containerID="cri-o://14a69174c80b43eedd3cf2d47e9002315412cb3c4399d84aa7fd71869445e64a" gracePeriod=30 Oct 30 13:16:16.599182 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.599046 5029 generic.go:356] "Generic (PLEG): container finished" podID="9b89f6e6-34f4-4888-bea4-26ed65d618e2" containerID="14a69174c80b43eedd3cf2d47e9002315412cb3c4399d84aa7fd71869445e64a" exitCode=0 Oct 30 13:16:16.599182 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.599104 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" event={"ID":"9b89f6e6-34f4-4888-bea4-26ed65d618e2","Type":"ContainerDied","Data":"14a69174c80b43eedd3cf2d47e9002315412cb3c4399d84aa7fd71869445e64a"} Oct 30 13:16:16.717565 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.717537 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:16:16.889081 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.889019 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9b89f6e6-34f4-4888-bea4-26ed65d618e2-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b89f6e6-34f4-4888-bea4-26ed65d618e2-local-certs\") pod \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " Oct 30 13:16:16.889287 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.889135 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-kubeconfig\") pod \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " Oct 30 13:16:16.889287 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.889181 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-token\" (UniqueName: \"kubernetes.io/projected/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-token\") pod \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " Oct 30 13:16:16.889287 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.889225 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9b89f6e6-34f4-4888-bea4-26ed65d618e2-kube-api-access-hqv2p\" (UniqueName: \"kubernetes.io/projected/9b89f6e6-34f4-4888-bea4-26ed65d618e2-kube-api-access-hqv2p\") pod \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " Oct 30 13:16:16.889287 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.889282 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-csr-ca-configmap\") pod \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " Oct 30 13:16:16.889527 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.889357 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9b89f6e6-34f4-4888-bea4-26ed65d618e2-cacerts\" (UniqueName: \"kubernetes.io/secret/9b89f6e6-34f4-4888-bea4-26ed65d618e2-cacerts\") pod \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " Oct 30 13:16:16.889527 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.889401 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-csr-dns-cert\") pod \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\" (UID: \"9b89f6e6-34f4-4888-bea4-26ed65d618e2\") " Oct 30 13:16:16.890939 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.890659 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-csr-ca-configmap" pod "9b89f6e6-34f4-4888-bea4-26ed65d618e2" (UID: "9b89f6e6-34f4-4888-bea4-26ed65d618e2"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:16:16.894216 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.894173 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-csr-dns-cert" pod "9b89f6e6-34f4-4888-bea4-26ed65d618e2" (UID: "9b89f6e6-34f4-4888-bea4-26ed65d618e2"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:16:16.894338 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.894280 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-kubeconfig" pod "9b89f6e6-34f4-4888-bea4-26ed65d618e2" (UID: "9b89f6e6-34f4-4888-bea4-26ed65d618e2"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:16:16.894857 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.894813 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b89f6e6-34f4-4888-bea4-26ed65d618e2-cacerts" pod "9b89f6e6-34f4-4888-bea4-26ed65d618e2" (UID: "9b89f6e6-34f4-4888-bea4-26ed65d618e2"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:16:16.895888 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.895854 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b89f6e6-34f4-4888-bea4-26ed65d618e2-kube-api-access-hqv2p" pod "9b89f6e6-34f4-4888-bea4-26ed65d618e2" (UID: "9b89f6e6-34f4-4888-bea4-26ed65d618e2"). InnerVolumeSpecName "kube-api-access-hqv2p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:16:16.896432 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.896388 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-token" pod "9b89f6e6-34f4-4888-bea4-26ed65d618e2" (UID: "9b89f6e6-34f4-4888-bea4-26ed65d618e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:16:16.896624 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.896598 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b89f6e6-34f4-4888-bea4-26ed65d618e2-local-certs" pod "9b89f6e6-34f4-4888-bea4-26ed65d618e2" (UID: "9b89f6e6-34f4-4888-bea4-26ed65d618e2"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 13:16:16.990632 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.990537 5029 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-kubeconfig\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:16:16.990632 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.990582 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-token\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:16:16.990632 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.990599 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqv2p\" (UniqueName: \"kubernetes.io/projected/9b89f6e6-34f4-4888-bea4-26ed65d618e2-kube-api-access-hqv2p\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:16:16.990632 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.990616 5029 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-csr-ca-configmap\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:16:16.990632 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.990630 5029 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9b89f6e6-34f4-4888-bea4-26ed65d618e2-cacerts\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:16:16.990912 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.990645 5029 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9b89f6e6-34f4-4888-bea4-26ed65d618e2-istio-csr-dns-cert\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:16:16.990912 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:16.990658 5029 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9b89f6e6-34f4-4888-bea4-26ed65d618e2-local-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:16:17.620924 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:17.620889 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" event={"ID":"9b89f6e6-34f4-4888-bea4-26ed65d618e2","Type":"ContainerDied","Data":"0dec5a6af3c61275a912c50760b03ec16313095261b06f040e34d6d078516bc1"} Oct 30 13:16:17.621571 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:17.621531 5029 scope.go:122] "RemoveContainer" containerID="14a69174c80b43eedd3cf2d47e9002315412cb3c4399d84aa7fd71869445e64a" Oct 30 13:16:17.621926 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:17.621895 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc" Oct 30 13:16:17.726089 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:17.724137 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc"] Oct 30 13:16:17.738975 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:17.738941 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m2qwc"] Oct 30 13:16:18.792248 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:18.792195 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-ftzzl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 13:16:18.792923 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:18.792892 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-ftzzl" podUID="1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 13:16:19.088334 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:19.088303 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9b89f6e6-34f4-4888-bea4-26ed65d618e2" path="/var/lib/kubelet/pods/9b89f6e6-34f4-4888-bea4-26ed65d618e2/volumes" Oct 30 13:16:41.255979 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:41.255950 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/2.log" Oct 30 13:16:41.259237 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:16:41.259215 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/2.log" Oct 30 13:16:44.878725 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:16:44.878691 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:17:14.885737 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:17:14.885682 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:17:16.879289 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:17:16.879249 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-5bc884dd87-lmc5v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 13:17:16.879925 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:17:16.879875 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-lmc5v" podUID="6a6dde7d-d204-46b0-b9fc-632872c1c634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 13:17:44.877859 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:17:44.877823 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:18:14.878584 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:18:14.878508 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:18:44.878491 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:18:44.878456 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:19:14.879271 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:19:14.879223 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:19:44.878029 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:19:44.877992 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:20:14.878338 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:20:14.878295 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:20:44.878297 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:20:44.878199 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:21:14.877524 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:21:14.877480 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:21:41.283869 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:21:41.283839 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/2.log" Oct 30 13:21:41.288538 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:21:41.288515 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/2.log" Oct 30 13:21:44.878541 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:21:44.878508 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:22:14.878412 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:22:14.878372 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:22:44.878552 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:22:44.878512 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:23:14.878547 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:23:14.878511 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:23:44.878361 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:23:44.878324 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:24:14.878381 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:24:14.878301 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:24:44.878200 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:24:44.878163 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:25:14.877912 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:25:14.877876 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:25:25.473408 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:25.473317 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-rfq8d"] Oct 30 13:25:25.473838 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:25.473693 5029 cpu_manager.go:414] "containerMap: removing container" podUID="9b89f6e6-34f4-4888-bea4-26ed65d618e2" containerName="discovery" Oct 30 13:25:25.473838 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:25.473705 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9b89f6e6-34f4-4888-bea4-26ed65d618e2" containerName="discovery" Oct 30 13:25:25.473838 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:25.473793 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="9b89f6e6-34f4-4888-bea4-26ed65d618e2" containerName="discovery" Oct 30 13:25:25.476633 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:25.476615 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-rfq8d" Oct 30 13:25:25.509293 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:25.509263 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-rfq8d"] Oct 30 13:25:25.571778 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:25.571749 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d3025a0-5363-422f-a77c-14a1eea1d942-metrics-certs\") pod \"maas-api-595cc648c-rfq8d\" (UID: \"5d3025a0-5363-422f-a77c-14a1eea1d942\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-rfq8d" Oct 30 13:25:25.571929 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:25.571784 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-769vf\" (UniqueName: \"kubernetes.io/projected/5d3025a0-5363-422f-a77c-14a1eea1d942-kube-api-access-769vf\") pod \"maas-api-595cc648c-rfq8d\" (UID: \"5d3025a0-5363-422f-a77c-14a1eea1d942\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-rfq8d" Oct 30 13:25:25.571929 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:25.571881 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d3025a0-5363-422f-a77c-14a1eea1d942-maas-api-tls\") pod \"maas-api-595cc648c-rfq8d\" (UID: \"5d3025a0-5363-422f-a77c-14a1eea1d942\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-rfq8d" Oct 30 13:25:25.572003 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:25.571949 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d3025a0-5363-422f-a77c-14a1eea1d942-openshift-service-ca\") pod \"maas-api-595cc648c-rfq8d\" (UID: \"5d3025a0-5363-422f-a77c-14a1eea1d942\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-rfq8d" Oct 30 13:25:25.673029 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:25.672992 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d3025a0-5363-422f-a77c-14a1eea1d942-metrics-certs\") pod \"maas-api-595cc648c-rfq8d\" (UID: \"5d3025a0-5363-422f-a77c-14a1eea1d942\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-rfq8d" Oct 30 13:25:25.673029 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:25.673030 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-769vf\" (UniqueName: \"kubernetes.io/projected/5d3025a0-5363-422f-a77c-14a1eea1d942-kube-api-access-769vf\") pod \"maas-api-595cc648c-rfq8d\" (UID: \"5d3025a0-5363-422f-a77c-14a1eea1d942\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-rfq8d" Oct 30 13:25:25.673250 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:25.673097 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d3025a0-5363-422f-a77c-14a1eea1d942-maas-api-tls\") pod \"maas-api-595cc648c-rfq8d\" (UID: \"5d3025a0-5363-422f-a77c-14a1eea1d942\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-rfq8d" Oct 30 13:25:25.673250 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:25.673165 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d3025a0-5363-422f-a77c-14a1eea1d942-openshift-service-ca\") pod \"maas-api-595cc648c-rfq8d\" (UID: \"5d3025a0-5363-422f-a77c-14a1eea1d942\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-rfq8d" Oct 30 13:25:25.673838 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:25.673816 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d3025a0-5363-422f-a77c-14a1eea1d942-openshift-service-ca\") pod \"maas-api-595cc648c-rfq8d\" (UID: \"5d3025a0-5363-422f-a77c-14a1eea1d942\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-rfq8d" Oct 30 13:25:25.675547 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:25.675524 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d3025a0-5363-422f-a77c-14a1eea1d942-metrics-certs\") pod \"maas-api-595cc648c-rfq8d\" (UID: \"5d3025a0-5363-422f-a77c-14a1eea1d942\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-rfq8d" Oct 30 13:25:25.675696 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:25.675675 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d3025a0-5363-422f-a77c-14a1eea1d942-maas-api-tls\") pod \"maas-api-595cc648c-rfq8d\" (UID: \"5d3025a0-5363-422f-a77c-14a1eea1d942\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-rfq8d" Oct 30 13:25:25.699086 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:25.699036 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-769vf\" (UniqueName: \"kubernetes.io/projected/5d3025a0-5363-422f-a77c-14a1eea1d942-kube-api-access-769vf\") pod \"maas-api-595cc648c-rfq8d\" (UID: \"5d3025a0-5363-422f-a77c-14a1eea1d942\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-rfq8d" Oct 30 13:25:25.789227 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:25.789156 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-rfq8d" Oct 30 13:25:26.103982 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:26.103957 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-rfq8d"] Oct 30 13:25:26.104699 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:25:26.104668 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d3025a0_5363_422f_a77c_14a1eea1d942.slice/crio-4cb9c5f7305f05f06dfb5bbe34db3a34318e549593ecb5554283962ea14dd30e WatchSource:0}: Error finding container 4cb9c5f7305f05f06dfb5bbe34db3a34318e549593ecb5554283962ea14dd30e: Status 404 returned error can't find the container with id 4cb9c5f7305f05f06dfb5bbe34db3a34318e549593ecb5554283962ea14dd30e Oct 30 13:25:26.106038 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:26.106021 5029 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 13:25:26.386093 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:26.385942 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-rfq8d" event={"ID":"5d3025a0-5363-422f-a77c-14a1eea1d942","Type":"ContainerStarted","Data":"4cb9c5f7305f05f06dfb5bbe34db3a34318e549593ecb5554283962ea14dd30e"} Oct 30 13:25:27.423091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:27.416969 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-rfq8d" event={"ID":"5d3025a0-5363-422f-a77c-14a1eea1d942","Type":"ContainerStarted","Data":"9f018ac3c256ed871e99bd6dec61e9cd1e69639fe5ce0b6c6581d5e7208ed886"} Oct 30 13:25:27.423091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:27.418049 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-rfq8d" Oct 30 13:25:27.468299 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:27.468249 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-595cc648c-rfq8d" podStartSLOduration=2.061391206 podStartE2EDuration="2.468236404s" totalImagesPullingTime="406.845198ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 13:25:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:25:27.467413153 +0000 UTC m=+2026.493054435" watchObservedRunningTime="2026-10-30 13:25:27.468236404 +0000 UTC m=+2026.493877643" Oct 30 13:25:34.437047 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:34.437013 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-rfq8d" Oct 30 13:25:34.516791 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:34.516748 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx"] Oct 30 13:25:34.517030 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:34.516999 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" podUID="89ee7c56-9d24-468e-a804-8729861ad1bd" containerName="maas-api" containerID="cri-o://d5baca55c310c9b051b2fc8be94dfc10cbfa89022c2e029471a56e3ce252a354" gracePeriod=30 Oct 30 13:25:36.041670 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:36.041342 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" Oct 30 13:25:36.049556 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:36.049522 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/89ee7c56-9d24-468e-a804-8729861ad1bd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ee7c56-9d24-468e-a804-8729861ad1bd-metrics-certs\") pod \"89ee7c56-9d24-468e-a804-8729861ad1bd\" (UID: \"89ee7c56-9d24-468e-a804-8729861ad1bd\") " Oct 30 13:25:36.049787 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:36.049767 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/89ee7c56-9d24-468e-a804-8729861ad1bd-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89ee7c56-9d24-468e-a804-8729861ad1bd-openshift-service-ca\") pod \"89ee7c56-9d24-468e-a804-8729861ad1bd\" (UID: \"89ee7c56-9d24-468e-a804-8729861ad1bd\") " Oct 30 13:25:36.049968 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:36.049949 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89ee7c56-9d24-468e-a804-8729861ad1bd-kube-api-access-jc5mh\" (UniqueName: \"kubernetes.io/projected/89ee7c56-9d24-468e-a804-8729861ad1bd-kube-api-access-jc5mh\") pod \"89ee7c56-9d24-468e-a804-8729861ad1bd\" (UID: \"89ee7c56-9d24-468e-a804-8729861ad1bd\") " Oct 30 13:25:36.050139 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:36.050121 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/89ee7c56-9d24-468e-a804-8729861ad1bd-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89ee7c56-9d24-468e-a804-8729861ad1bd-maas-api-tls\") pod \"89ee7c56-9d24-468e-a804-8729861ad1bd\" (UID: \"89ee7c56-9d24-468e-a804-8729861ad1bd\") " Oct 30 13:25:36.050252 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:36.050015 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89ee7c56-9d24-468e-a804-8729861ad1bd-openshift-service-ca" pod "89ee7c56-9d24-468e-a804-8729861ad1bd" (UID: "89ee7c56-9d24-468e-a804-8729861ad1bd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 13:25:36.050578 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:36.050564 5029 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89ee7c56-9d24-468e-a804-8729861ad1bd-openshift-service-ca\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:25:36.052417 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:36.052384 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89ee7c56-9d24-468e-a804-8729861ad1bd-kube-api-access-jc5mh" pod "89ee7c56-9d24-468e-a804-8729861ad1bd" (UID: "89ee7c56-9d24-468e-a804-8729861ad1bd"). InnerVolumeSpecName "kube-api-access-jc5mh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 13:25:36.052504 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:36.052446 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89ee7c56-9d24-468e-a804-8729861ad1bd-metrics-certs" pod "89ee7c56-9d24-468e-a804-8729861ad1bd" (UID: "89ee7c56-9d24-468e-a804-8729861ad1bd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:25:36.052657 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:36.052642 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89ee7c56-9d24-468e-a804-8729861ad1bd-maas-api-tls" pod "89ee7c56-9d24-468e-a804-8729861ad1bd" (UID: "89ee7c56-9d24-468e-a804-8729861ad1bd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 13:25:36.151744 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:36.151660 5029 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89ee7c56-9d24-468e-a804-8729861ad1bd-metrics-certs\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:25:36.151744 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:36.151697 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jc5mh\" (UniqueName: \"kubernetes.io/projected/89ee7c56-9d24-468e-a804-8729861ad1bd-kube-api-access-jc5mh\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:25:36.151744 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:36.151714 5029 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89ee7c56-9d24-468e-a804-8729861ad1bd-maas-api-tls\") on node \"ip-10-0-141-147.ec2.internal\" DevicePath \"\"" Oct 30 13:25:36.538479 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:36.538415 5029 generic.go:356] "Generic (PLEG): container finished" podID="89ee7c56-9d24-468e-a804-8729861ad1bd" containerID="d5baca55c310c9b051b2fc8be94dfc10cbfa89022c2e029471a56e3ce252a354" exitCode=0 Oct 30 13:25:36.538834 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:36.538805 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" event={"ID":"89ee7c56-9d24-468e-a804-8729861ad1bd","Type":"ContainerDied","Data":"d5baca55c310c9b051b2fc8be94dfc10cbfa89022c2e029471a56e3ce252a354"} Oct 30 13:25:36.539029 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:36.539007 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" event={"ID":"89ee7c56-9d24-468e-a804-8729861ad1bd","Type":"ContainerDied","Data":"048d0fe92b9b407fae5e6c8bb01d89abd0302c179a9641a07737f30dd764f47d"} Oct 30 13:25:36.539214 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:36.539196 5029 scope.go:122] "RemoveContainer" containerID="d5baca55c310c9b051b2fc8be94dfc10cbfa89022c2e029471a56e3ce252a354" Oct 30 13:25:36.539394 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:36.539195 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx" Oct 30 13:25:36.553987 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:36.553956 5029 scope.go:122] "RemoveContainer" containerID="d5baca55c310c9b051b2fc8be94dfc10cbfa89022c2e029471a56e3ce252a354" Oct 30 13:25:36.554400 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:25:36.554371 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5baca55c310c9b051b2fc8be94dfc10cbfa89022c2e029471a56e3ce252a354\": container with ID starting with d5baca55c310c9b051b2fc8be94dfc10cbfa89022c2e029471a56e3ce252a354 not found: ID does not exist" containerID="d5baca55c310c9b051b2fc8be94dfc10cbfa89022c2e029471a56e3ce252a354" Oct 30 13:25:36.554481 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:36.554409 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5baca55c310c9b051b2fc8be94dfc10cbfa89022c2e029471a56e3ce252a354"} err="failed to get container status \"d5baca55c310c9b051b2fc8be94dfc10cbfa89022c2e029471a56e3ce252a354\": rpc error: code = NotFound desc = could not find container \"d5baca55c310c9b051b2fc8be94dfc10cbfa89022c2e029471a56e3ce252a354\": container with ID starting with d5baca55c310c9b051b2fc8be94dfc10cbfa89022c2e029471a56e3ce252a354 not found: ID does not exist" Oct 30 13:25:36.627090 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:36.615717 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx"] Oct 30 13:25:36.632103 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:36.631407 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-zg7sx"] Oct 30 13:25:37.090222 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:25:37.090184 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="89ee7c56-9d24-468e-a804-8729861ad1bd" path="/var/lib/kubelet/pods/89ee7c56-9d24-468e-a804-8729861ad1bd/volumes" Oct 30 13:25:44.878513 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:25:44.878412 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:26:14.878107 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:26:14.878044 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:26:27.045364 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:27.045333 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-7flkr_218ec9c4-47f2-4fe7-82e2-f34b64920e7d/manager/0.log" Oct 30 13:26:27.253951 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:27.253921 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-5cfb77bf8c-b7tkl_9187d211-b3d1-4945-bb8b-90593c940415/manager/0.log" Oct 30 13:26:27.471536 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:27.471508 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-c27pr_aa264999-e5bc-4ff5-83c8-0cc23321891c/manager/0.log" Oct 30 13:26:28.086688 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:28.086665 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-t2hdx_793ac5fe-0be7-457b-bb46-98c39361bbe5/server/0.log" Oct 30 13:26:29.223943 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:29.223915 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw_404322af-4f87-4eb0-909e-6d2bb2cba6b5/copy-manifests/0.log" Oct 30 13:26:29.261091 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:29.261050 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw_404322af-4f87-4eb0-909e-6d2bb2cba6b5/manager/0.log" Oct 30 13:26:29.496650 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:29.496581 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-7d94f55f4b-jr9k6_5029dd68-f246-4a21-8181-f5a5dbb9f574/proxy/0.log" Oct 30 13:26:30.485521 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:30.485489 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-rfq8d_5d3025a0-5363-422f-a77c-14a1eea1d942/maas-api/0.log" Oct 30 13:26:34.427593 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:34.427561 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-gjmnr_43885517-71fe-49a2-86fe-1cb4ee7fbb6b/kuadrant-console-plugin/0.log" Oct 30 13:26:35.841499 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:35.841468 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6455b4886fxpfrt_70776581-8a8f-4d3a-8454-0e6e251c3ce2/istio-proxy/0.log" Oct 30 13:26:37.044534 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:37.044486 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-9h8z8_11ce4393-c9ba-4c45-a9e8-276e334f6be0/istio-proxy/0.log" Oct 30 13:26:37.653638 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:37.653610 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-ftzzl_1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b/istio-proxy/0.log" Oct 30 13:26:38.268375 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:38.268323 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-lmc5v_6a6dde7d-d204-46b0-b9fc-632872c1c634/istio-proxy/1.log" Oct 30 13:26:39.054234 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:39.054201 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-767cffcd47-qx67t_3e77ccc1-14a3-4f85-a214-854bd392733a/router/0.log" Oct 30 13:26:39.495746 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:39.495717 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-lmc5v_6a6dde7d-d204-46b0-b9fc-632872c1c634/istio-proxy/0.log" Oct 30 13:26:41.314215 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:41.314187 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/2.log" Oct 30 13:26:41.318631 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:41.318608 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/2.log" Oct 30 13:26:44.878614 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:26:44.878572 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:26:51.213829 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:51.213796 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-7flkr_218ec9c4-47f2-4fe7-82e2-f34b64920e7d/manager/0.log" Oct 30 13:26:51.433731 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:51.433700 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-5cfb77bf8c-b7tkl_9187d211-b3d1-4945-bb8b-90593c940415/manager/0.log" Oct 30 13:26:51.646035 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:51.646006 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-c27pr_aa264999-e5bc-4ff5-83c8-0cc23321891c/manager/0.log" Oct 30 13:26:52.109670 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:52.109638 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-t2hdx_793ac5fe-0be7-457b-bb46-98c39361bbe5/server/0.log" Oct 30 13:26:53.222362 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:53.222333 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw_404322af-4f87-4eb0-909e-6d2bb2cba6b5/copy-manifests/0.log" Oct 30 13:26:53.268269 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:53.268239 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7bb4ff5b5c-r4ksw_404322af-4f87-4eb0-909e-6d2bb2cba6b5/manager/0.log" Oct 30 13:26:53.507835 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:53.507744 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-7d94f55f4b-jr9k6_5029dd68-f246-4a21-8181-f5a5dbb9f574/proxy/0.log" Oct 30 13:26:54.257140 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:54.257114 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-rfq8d_5d3025a0-5363-422f-a77c-14a1eea1d942/maas-api/0.log" Oct 30 13:26:57.936603 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:57.936574 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-gjmnr_43885517-71fe-49a2-86fe-1cb4ee7fbb6b/kuadrant-console-plugin/0.log" Oct 30 13:26:59.147747 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:26:59.147721 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6455b4886fxpfrt_70776581-8a8f-4d3a-8454-0e6e251c3ce2/istio-proxy/0.log" Oct 30 13:27:00.294198 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:27:00.294167 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-9h8z8_11ce4393-c9ba-4c45-a9e8-276e334f6be0/istio-proxy/0.log" Oct 30 13:27:00.906344 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:27:00.906306 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-ftzzl_1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b/istio-proxy/0.log" Oct 30 13:27:01.529885 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:27:01.529853 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-lmc5v_6a6dde7d-d204-46b0-b9fc-632872c1c634/istio-proxy/1.log" Oct 30 13:27:02.280885 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:27:02.280851 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-767cffcd47-qx67t_3e77ccc1-14a3-4f85-a214-854bd392733a/router/0.log" Oct 30 13:27:14.878813 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:27:14.878775 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:27:44.877984 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:27:44.877947 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:28:14.878312 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:28:14.878268 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:28:29.934949 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:29.934735 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-lf7b2_75bd9ac5-49ff-4c58-85c7-79b181150555/global-pull-secret-syncer/0.log" Oct 30 13:28:30.516152 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:30.516126 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-nprbr_2f5f56fc-d2d4-4cef-86c0-0ac4b7a51e41/connection-checker/0.log" Oct 30 13:28:30.650264 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:30.650234 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-nzp7p_7d93802e-f47d-4319-b308-04fe84901551/connection-checker/0.log" Oct 30 13:28:30.778874 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:30.778798 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-wsfvm_1734d386-0cb1-4002-a954-b76b3cf62eaf/connection-checker/0.log" Oct 30 13:28:31.218755 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:31.218725 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vjn55_49b68250-16ee-42f1-9730-e4d326c96cd8/konnectivity-agent/0.log" Oct 30 13:28:31.641374 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:31.641343 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-147.ec2.internal_7d0024b3080a170827a29027ec64e58f/haproxy/0.log" Oct 30 13:28:38.145035 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:38.145003 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-gjmnr_43885517-71fe-49a2-86fe-1cb4ee7fbb6b/kuadrant-console-plugin/0.log" Oct 30 13:28:39.125538 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.125508 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk"] Oct 30 13:28:39.125876 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.125864 5029 cpu_manager.go:414] "containerMap: removing container" podUID="89ee7c56-9d24-468e-a804-8729861ad1bd" containerName="maas-api" Oct 30 13:28:39.125925 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.125877 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89ee7c56-9d24-468e-a804-8729861ad1bd" containerName="maas-api" Oct 30 13:28:39.125992 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.125974 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="89ee7c56-9d24-468e-a804-8729861ad1bd" containerName="maas-api" Oct 30 13:28:39.129001 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.128985 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk" Oct 30 13:28:39.149615 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.149591 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk"] Oct 30 13:28:39.237947 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.237916 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z6jh4\"/\"kube-root-ca.crt\"" Oct 30 13:28:39.237947 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.237932 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z6jh4\"/\"openshift-service-ca.crt\"" Oct 30 13:28:39.238158 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.237982 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-z6jh4\"/\"default-dockercfg-hfvsn\"" Oct 30 13:28:39.239350 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.239332 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnwvj\" (UniqueName: \"kubernetes.io/projected/7cff2baf-9431-474d-b523-6578b78d5ae1-kube-api-access-wnwvj\") pod \"perf-node-gather-daemonset-znksk\" (UID: \"7cff2baf-9431-474d-b523-6578b78d5ae1\") " pod="openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk" Oct 30 13:28:39.239399 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.239360 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7cff2baf-9431-474d-b523-6578b78d5ae1-sys\") pod \"perf-node-gather-daemonset-znksk\" (UID: \"7cff2baf-9431-474d-b523-6578b78d5ae1\") " pod="openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk" Oct 30 13:28:39.239399 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.239382 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7cff2baf-9431-474d-b523-6578b78d5ae1-proc\") pod \"perf-node-gather-daemonset-znksk\" (UID: \"7cff2baf-9431-474d-b523-6578b78d5ae1\") " pod="openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk" Oct 30 13:28:39.239506 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.239455 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7cff2baf-9431-474d-b523-6578b78d5ae1-lib-modules\") pod \"perf-node-gather-daemonset-znksk\" (UID: \"7cff2baf-9431-474d-b523-6578b78d5ae1\") " pod="openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk" Oct 30 13:28:39.239546 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.239522 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7cff2baf-9431-474d-b523-6578b78d5ae1-podres\") pod \"perf-node-gather-daemonset-znksk\" (UID: \"7cff2baf-9431-474d-b523-6578b78d5ae1\") " pod="openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk" Oct 30 13:28:39.340587 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.340552 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7cff2baf-9431-474d-b523-6578b78d5ae1-podres\") pod \"perf-node-gather-daemonset-znksk\" (UID: \"7cff2baf-9431-474d-b523-6578b78d5ae1\") " pod="openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk" Oct 30 13:28:39.340750 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.340651 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnwvj\" (UniqueName: \"kubernetes.io/projected/7cff2baf-9431-474d-b523-6578b78d5ae1-kube-api-access-wnwvj\") pod \"perf-node-gather-daemonset-znksk\" (UID: \"7cff2baf-9431-474d-b523-6578b78d5ae1\") " pod="openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk" Oct 30 13:28:39.340750 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.340678 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7cff2baf-9431-474d-b523-6578b78d5ae1-sys\") pod \"perf-node-gather-daemonset-znksk\" (UID: \"7cff2baf-9431-474d-b523-6578b78d5ae1\") " pod="openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk" Oct 30 13:28:39.340750 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.340684 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7cff2baf-9431-474d-b523-6578b78d5ae1-podres\") pod \"perf-node-gather-daemonset-znksk\" (UID: \"7cff2baf-9431-474d-b523-6578b78d5ae1\") " pod="openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk" Oct 30 13:28:39.340750 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.340711 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7cff2baf-9431-474d-b523-6578b78d5ae1-proc\") pod \"perf-node-gather-daemonset-znksk\" (UID: \"7cff2baf-9431-474d-b523-6578b78d5ae1\") " pod="openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk" Oct 30 13:28:39.340750 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.340735 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7cff2baf-9431-474d-b523-6578b78d5ae1-sys\") pod \"perf-node-gather-daemonset-znksk\" (UID: \"7cff2baf-9431-474d-b523-6578b78d5ae1\") " pod="openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk" Oct 30 13:28:39.340750 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.340743 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7cff2baf-9431-474d-b523-6578b78d5ae1-lib-modules\") pod \"perf-node-gather-daemonset-znksk\" (UID: \"7cff2baf-9431-474d-b523-6578b78d5ae1\") " pod="openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk" Oct 30 13:28:39.341055 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.340768 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7cff2baf-9431-474d-b523-6578b78d5ae1-proc\") pod \"perf-node-gather-daemonset-znksk\" (UID: \"7cff2baf-9431-474d-b523-6578b78d5ae1\") " pod="openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk" Oct 30 13:28:39.341055 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.340865 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7cff2baf-9431-474d-b523-6578b78d5ae1-lib-modules\") pod \"perf-node-gather-daemonset-znksk\" (UID: \"7cff2baf-9431-474d-b523-6578b78d5ae1\") " pod="openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk" Oct 30 13:28:39.366654 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.366629 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnwvj\" (UniqueName: \"kubernetes.io/projected/7cff2baf-9431-474d-b523-6578b78d5ae1-kube-api-access-wnwvj\") pod \"perf-node-gather-daemonset-znksk\" (UID: \"7cff2baf-9431-474d-b523-6578b78d5ae1\") " pod="openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk" Oct 30 13:28:39.441819 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.441743 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk" Oct 30 13:28:39.741579 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:39.741551 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk"] Oct 30 13:28:39.742722 ip-10-0-141-147 kubenswrapper[5029]: W1030 13:28:39.742696 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7cff2baf_9431_474d_b523_6578b78d5ae1.slice/crio-b2576f84647ecdbea356c7743954ff3dbab542e61b9dd76f5a8277d19fd4d8f2 WatchSource:0}: Error finding container b2576f84647ecdbea356c7743954ff3dbab542e61b9dd76f5a8277d19fd4d8f2: Status 404 returned error can't find the container with id b2576f84647ecdbea356c7743954ff3dbab542e61b9dd76f5a8277d19fd4d8f2 Oct 30 13:28:40.408232 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:40.408192 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk" event={"ID":"7cff2baf-9431-474d-b523-6578b78d5ae1","Type":"ContainerStarted","Data":"0dcfdd5f8ed0e10cfa013f91633a60ecd2d9e149882dbf779948f3429170b6f2"} Oct 30 13:28:40.408628 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:40.408256 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk" event={"ID":"7cff2baf-9431-474d-b523-6578b78d5ae1","Type":"ContainerStarted","Data":"b2576f84647ecdbea356c7743954ff3dbab542e61b9dd76f5a8277d19fd4d8f2"} Oct 30 13:28:40.408628 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:40.408338 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk" Oct 30 13:28:40.481296 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:40.481247 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk" podStartSLOduration=1.48123241 podStartE2EDuration="1.48123241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 13:28:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 13:28:40.479732513 +0000 UTC m=+2219.505373760" watchObservedRunningTime="2026-10-30 13:28:40.48123241 +0000 UTC m=+2219.506873669" Oct 30 13:28:40.841230 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:40.841179 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-6s9dp_f0d51a79-823b-4059-94c3-0535fc882a44/cluster-monitoring-operator/0.log" Oct 30 13:28:42.257941 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:42.257915 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zwmzb_5a0202a8-58b5-4cb3-82b4-697263f11ad9/node-exporter/0.log" Oct 30 13:28:42.369367 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:42.369337 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zwmzb_5a0202a8-58b5-4cb3-82b4-697263f11ad9/kube-rbac-proxy/0.log" Oct 30 13:28:42.481997 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:42.481965 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zwmzb_5a0202a8-58b5-4cb3-82b4-697263f11ad9/init-textfile/0.log" Oct 30 13:28:44.878305 ip-10-0-141-147 kubenswrapper[5029]: E1030 13:28:44.878263 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-l7z89" containerName="extractor" Oct 30 13:28:46.422579 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:46.422551 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-z6jh4/perf-node-gather-daemonset-znksk" Oct 30 13:28:46.429130 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:46.429111 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-86c478bb88-49h5k_109e0336-ecdb-4278-b5d3-72c7bdb80276/networking-console-plugin/0.log" Oct 30 13:28:47.496295 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:47.496258 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/2.log" Oct 30 13:28:47.499858 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:47.499835 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-nbwj8_fdddab43-acaf-4153-bb52-3e60044a35dc/console-operator/3.log" Oct 30 13:28:49.700533 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:49.700501 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-58c9f5f46d-bzh2m_1a8f7b67-6d17-4f5e-a79b-52481ea0fd48/volume-data-source-validator/0.log" Oct 30 13:28:52.186494 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:52.186467 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cd6w9_38b6bfaf-2fb2-455b-9e7d-32586b3f216f/dns/0.log" Oct 30 13:28:52.255997 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:52.255967 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6455b4886fxpfrt_70776581-8a8f-4d3a-8454-0e6e251c3ce2/istio-proxy/0.log" Oct 30 13:28:52.316914 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:52.316888 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cd6w9_38b6bfaf-2fb2-455b-9e7d-32586b3f216f/kube-rbac-proxy/0.log" Oct 30 13:28:52.677349 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:52.677320 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-759xk_7fdf07a3-cf6b-4fa7-9b90-6e92257d5d0d/dns-node-resolver/0.log" Oct 30 13:28:54.030181 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:54.030154 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-9h8z8_11ce4393-c9ba-4c45-a9e8-276e334f6be0/istio-proxy/0.log" Oct 30 13:28:54.237143 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:54.237116 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-ftzzl_1e1c1b8f-cd7b-44dc-b55a-cbd7c894a38b/istio-proxy/0.log" Oct 30 13:28:54.282179 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:54.282111 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vsf9w_2a12192e-f0c8-4354-b9fa-1b65b04ebc19/node-ca/0.log" Oct 30 13:28:54.439725 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:54.439699 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-lmc5v_6a6dde7d-d204-46b0-b9fc-632872c1c634/istio-proxy/0.log" Oct 30 13:28:54.440652 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:54.440633 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5bc884dd87-lmc5v_6a6dde7d-d204-46b0-b9fc-632872c1c634/istio-proxy/1.log" Oct 30 13:28:54.679313 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:54.679279 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-767cffcd47-qx67t_3e77ccc1-14a3-4f85-a214-854bd392733a/router/0.log" Oct 30 13:28:56.153752 ip-10-0-141-147 kubenswrapper[5029]: I1030 13:28:56.153722 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6455b4886fxpfrt_70776581-8a8f-4d3a-8454-0e6e251c3ce2/istio-proxy/0.log"